DeepSea deployment results in errors for the iSCSI Gateway

This document (7018668) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Enterprise Storage 4

Situation

When deploying SUSE Enterprise Storage using Salt/DeepSea and having an iSCSI Gateway configured, the following errors are seen when running stage 4:

:~ # salt-run state.orch ceph.stage.4
[WARNING ] Could not write out jid file for job 20170221120717480962. Retrying.
[WARNING ] Could not write out jid file for job 20170221120717480962. Retrying.
[WARNING ] Could not write out jid file for job 20170221120717480962. Retrying.
[WARNING ] Could not write out jid file for job 20170221120717480962. Retrying.
[WARNING ] Could not write out jid file for job 20170221120717480962. Retrying.
[ERROR   ] prep_jid could not store a jid after 5 tries.
[ERROR   ] Could not store job cache info. Job details for this run may be unavailable.
[ERROR   ] Run failed on minions: server.lab.suse.de
Failures:
    server.lab.suse.de:
      Name: lrbd - Function: pkg.installed - Result: Clean Started: - 12:07:27.126075 Duration: 484.869 ms
    ----------
              ID: enable lrbd
        Function: service.running
            Name: lrbd
          Result: False
         Comment: Service lrbd has been enabled, and is dead
         Started: 12:07:27.611338
        Duration: 894.987 ms
         Changes:   
                  ----------
                  lrbd:
                      True
      Name: service.restart - Function: module.run - Result: Changed Started: - 12:07:28.506752 Duration: 1359.962 ms
    
    Summary for server.lab.suse.de                                                                                                                                    
    ------------                                                                                                                                                           
    Succeeded: 2 (changed=2)
    Failed:    1
    ------------
    Total states run:     3                                                                                                                                                
    Total run time:   2.740 s

No minions matched the target. No command was sent, no jid was assigned.
adminserver.lab.suse.de_master:
  Name: auth - Function: salt.state - Result: Changed Started: - 12:07:18.813741 Duration: 1248.03 ms
  Name: keyring - Function: salt.state - Result: Changed Started: - 12:07:20.061930 Duration: 255.429 ms
  Name: sysconfig - Function: salt.state - Result: Changed Started: - 12:07:20.317510 Duration: 358.052 ms
  Name: iscsi import - Function: salt.state - Result: Changed Started: - 12:07:20.675734 Duration: 6089.31 ms
----------
          ID: iscsi apply
    Function: salt.state
      Result: False
     Comment: Run failed on minions: server.lab.suse.de
              Failures:                                                                                                                                                    
                  server.lab.suse.de:                                                                                                                                 
                    Name: lrbd - Function: pkg.installed - Result: Clean Started: - 12:07:27.126075 Duration: 484.869 ms
                  ----------
                            ID: enable lrbd
                      Function: service.running
                          Name: lrbd
                        Result: False
                       Comment: Service lrbd has been enabled, and is dead
                       Started: 12:07:27.611338
                      Duration: 894.987 ms
                       Changes:   
                                ----------
                                lrbd:
                                    True
                    Name: service.restart - Function: module.run - Result: Changed Started: - 12:07:28.506752 Duration: 1359.962 ms
                  
                  Summary for server.lab.suse.de                                                                                                                      
                  ------------                                                                                                                                             
                  Succeeded: 2 (changed=2)
                  Failed:    1
                  ------------
                  Total states run:     3
                  Total run time:   2.740 s
     Started: 12:07:26.765193
    Duration: 3112.397 ms
     Changes:   
  Name: cephfs pools - Function: salt.state - Result: Changed Started: - 12:07:29.877742 Duration: 3778.575 ms
  Name: mds auth - Function: salt.state - Result: Changed Started: - 12:07:33.656472 Duration: 1072.477 ms
  Name: mds - Function: salt.state - Result: Changed Started: - 12:07:34.729093 Duration: 525.992 ms
  Name: rgw auth - Function: salt.state - Result: Clean Started: - 12:07:35.255234 Duration: 821.704 ms
  Name: rgw - Function: salt.state - Result: Clean Started: - 12:07:36.077085 Duration: 104.533 ms
  Name: openattic auth - Function: salt.state - Result: Changed Started: - 12:07:36.181747 Duration: 502.064 ms
  Name: openattic - Function: salt.state - Result: Changed Started: - 12:07:36.683972 Duration: 37952.848 ms
  Name: openattic keyring - Function: salt.state - Result: Changed Started: - 12:08:14.636991 Duration: 517.841 ms
  Name: openattic oaconfig - Function: salt.state - Result: Changed Started: - 12:08:15.155038 Duration: 13705.437 ms

Summary for adminserver.lab.suse.de_master
-------------
Succeeded: 13 (changed=11)
Failed:     1
-------------
Total states run:     14
Total run time:   70.045 s

Resolution

The errors are cosmetic and can be ignored.

Cause


Additional Information

The errors are more often encountered when running the Admin node as a virtual machine. To verify the configuration was successful run only the iSCSI specific subset (that forms part of Stage 4) again:

salt-run state.orch ceph.stage.iscsi

This should return something like the following:

:~ # salt-run state.orch ceph.stage.iscsi
adminserver.lab.suse.de_master:
  Name: auth - Function: salt.state - Result: Changed Started: - 17:26:33.706143 Duration: 1090.991 ms
  Name: keyring - Function: salt.state - Result: Clean Started: - 17:26:34.797278 Duration: 356.39 ms
  Name: sysconfig - Function: salt.state - Result: Changed Started: - 17:26:35.153830 Duration: 252.811 ms
  Name: iscsi import - Function: salt.state - Result: Changed Started: - 17:26:35.406796 Duration: 5123.672 ms
  Name: iscsi apply - Function: salt.state - Result: Changed Started: - 17:26:40.530622 Duration: 6884.202 ms

Summary for adminserver.hwlab.suse.de_master
------------
Succeeded: 5 (changed=4)
Failed:    0
------------
Total states run:     5
Total run time:  13.708 s

Alternatively to manually verify the installation take the following steps on the node designated to run the iSCSI Gateway:

1. Verify the lrbd service is running:

:~ # systemctl status lrbd.service 
● lrbd.service - configures target.service from Ceph
   Loaded: loaded (/usr/lib/systemd/system/lrbd.service; enabled; vendor preset: disabled)
   Active: active (exited) since Tue 2017-02-21 12:07:29 CET; 2h 55min ago
  Process: 32216 ExecStart=/usr/sbin/lrbd $LRBD_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 32216 (code=exited, status=0/SUCCESS)
    Tasks: 0 (limit: 512)
   CGroup: /system.slice/lrbd.service

2. Verify the configured RBD images were mapped, by default and with the below example an image demo is created:

:~ # rbd showmapped
id pool image snap device    
0  rbd  demo  -    /dev/rbd0

3. Verify the iSCSI Gateway is listening (the second option uses the default target port):

:~ # netstat -plt | grep iscsi
tcp        0      0 server.:iscsi-target *:*                     LISTEN      -         
          
:~ # netstat -plnt | grep 3260
tcp        0      0 192.100.100.10:3260      0.0.0.0:*               LISTEN      -  

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.

  • Document ID:7018668
  • Creation Date: 27-Feb-2017
  • Modified Date:03-Mar-2020
    • SUSE Enterprise Storage

< Back to Support Search

For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback@suse.com

SUSE Support Forums

Get your questions answered by experienced Sys Ops or interact with other SUSE community experts.

Join Our Community

Support Resources

Learn how to get the most from the technical support you receive with your SUSE Subscription, Premium Support, Academic Program, or Partner Program.


SUSE Customer Support Quick Reference Guide SUSE Technical Support Handbook Update Advisories
Support FAQ

Open an Incident

Open an incident with SUSE Technical Support, manage your subscriptions, download patches, or manage user access.

Go to Customer Center