driver compatable with truenas 12?

[marc5feb]

Hy,

i have installed a ussuri openstack with a truenas 12 storage system. The problem i face is that i can't connect with this cinder driver to my storage unit. As a workaround i downgrade to 11.3 update 5 freenas and now it works like a charm. i havent made any changes to my openstack configs for it to work.

this is an output of my cinder volume log.
2020-11-25 12:26:29.193 37975 INFO cinder.volume.manager [req-d55ea4c0-4095-47ad-944a-e8cc4b1cebc8 - - - - -] Starting volume driver FreeNASISCSIDriver (2.0.0)
2020-11-25 12:26:29.193 37975 INFO cinder.volume.drivers.ixsystems.iscsi [req-d55ea4c0-4095-47ad-944a-e8cc4b1cebc8 - - - - -] iXsystems Do Setup
2020-11-25 12:26:29.194 37975 INFO cinder.volume.drivers.ixsystems.iscsi [req-d55ea4c0-4095-47ad-944a-e8cc4b1cebc8 - - - - -] iXSystems: Check For Setup Error
2020-11-25 12:26:29.194 37975 INFO cinder.volume.drivers.ixsystems.iscsi [req-d55ea4c0-4095-47ad-944a-e8cc4b1cebc8 - - - - -] iXSystems: Check For Setup Error
2020-11-25 12:26:29.195 37975 INFO cinder.volume.drivers.ixsystems.iscsi [req-d55ea4c0-4095-47ad-944a-e8cc4b1cebc8 - - - - -] iXsystems Get Volume Status
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service [req-d55ea4c0-4095-47ad-944a-e8cc4b1cebc8 - - - - -] Error starting thread.: simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service Traceback (most recent call last):
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_service/service.py", line 810, in run_service
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service service.start()
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/cinder/service.py", line 228, in start
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service self.manager.init_host(added_to_cluster=self.added_to_cluster,
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/cinder/volume/manager.py", line 443, in init_host
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service self._init_host(added_to_cluster, **kwargs)
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/cinder/volume/manager.py", line 477, in _init_host
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service self.driver.init_capabilities()
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/cinder/volume/driver.py", line 770, in init_capabilities
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service stats = self.get_volume_stats(True)
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/cinder/volume/drivers/ixsystems/iscsi.py", line 199, in get_volume_stats
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service self.stats = self.common._update_volume_stats()
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/cinder/volume/drivers/ixsystems/common.py", line 368, in _update_volume_stats
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service data['total_capacity_gb'] = ix_utils.get_size_in_gb(json.loads(ret['response'])['avail'] + json.loads(ret['response'])['used'])
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/simplejson/init.py", line 518, in loads
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service return _default_decoder.decode(s)
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 370, in decode
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service obj, end = self.raw_decode(s)
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 400, in raw_decode
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service return self.scan_once(s, idx=_w(s, idx).end())
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
2020-11-25 12:26:29.199 37975 ERROR oslo_service.service
2020-11-25 12:26:29.209 37890 INFO oslo_service.service [req-40937f36-8916-4ef8-a741-45301f10732f - - - - -] Child 37975 exited with status 1
2020-11-25 12:26:29.218 37976 INFO cinder.service [-] Starting cinder-volume node (version 16.2.0)

has someone else tried it and got the same error? i have fully installed my openstack from scratch back then and i got the same error 2 times.

my cinder conf is

[ixsystems-iscsi]
iscsi_helper = tgtadm
volume_dd_blocksize = 512
volume_driver = cinder.volume.drivers.ixsystems.iscsi.FreeNASISCSIDriver
ixsystems_login = someloginname
ixsystems_password = somepassword
ixsystems_server_hostname = someipadress
ixsystems_volume_backend_name = iXsystems_FREENAS_Storage
ixsystems_iqn_prefix = iqn.2005-10.org.freenas.ctl
ixsystems_datastore_pool = Tank1
ixsystems_dataset_path = Tank1/XS-SH-ISCSI
ixsystems_vendor_name = iXsystems
ixsystems_storage_protocol = iscsi
#volume_backend_name = ixsystems-iscsi

[william-gr]

Are you using the master version of this repo?

[marc5feb]

Hy, The one downloaded and installed with git clone --depth=1 https://github.com/iXsystems/cinder cp -R ./cinder/driver/ixsystems/ /usr/lib/python3/dist-packages/cinder/volume/drivers/ Greetings, Marc Are you using the master version of this repo?

[william-gr]

Doesn't ring a bell, I have not been using it myself. Looks like the API reply was empty or invalid.

Would need happen from somebody to debug it

[dmcgrandle]

Looks like the error is happening when attempting to get volume stats. William, if you'll recall when I updated this driver to use the new v2 API we had an issue where the volume stats were only available using the older 1.0 API, so that one portion of this driver still relies on the older API. Is it possible that with truenas 12 support for the older API has been taken out and so that call is failing? I haven't installed 12 yet myself for testing ...

[william-gr]

Ah yes. 12.0 removes support for v1.0 API

[marc5feb]

thanks for the confirmation. will there be a new version which adresses this issue?

[dmcgrandle]

I am guessing that will have to wait until I have time to tackle it in the new year. I should have time in January to install a new test environment with 12 and update the code. If anyone else wants to tackle this in the meantime I have some documentation I generated for the 1.0 -> 2.0 upgrade I can share that will help.

[adasauce]

@dmcgrandle is would be swell if you could share the 1.0 -> 2.0 upgrade notes.

[adasauce]

this commit shosti@b4aabdf by @shosti appears to enable the 2.0 API for volume stats, and as well has another commit in their repo to handle API keys instead of using root username/password.

[shosti]

Oh yeah I think I got it working with API v2 on https://github.com/shosti/cinder/tree/api-key . Never got it up and running to the extent that I could verify that it works end-to-end which is why I didn’t open a PR, but I’m happy opening a PR if someone wants to test it.

[adasauce]

I'm in the process of testing out your branch right now, I'll let you know how the integration goes -- at least as an additional data point for whether or not it works as expected.

[adasauce]

@shosti at the most basic level, it's working 100% so far. stats work, volume create/delete works.

I'll be running it through it's paces a little more over the next few days.

[dmcgrandle]

Good work, and this is very interesting. A couple of comments:

• The @shosti code currently gets stats at the pool level. This isn't wrong per se, but if quotas have been assigned for the particular dataset then the stats returned will not be reflective of how much space is actually available to the driver for creation of new volumes. I'd suggest the polling be done at the dataset level.
• Very minor point - this code currently adds more variables than necessary and could be done in two changed lines instead of 5 new ones.
• I would separate this into two PRs, one for the code changes to _update_volume_stats() and another for the use of API keys.

With these points in mind, the updated code for update_volume_stats() would look like this:

 def _update_volume_stats(self):
    """Retrieve stats info from volume group
        REST API: $ GET /pools/mypool "size":95,"allocated":85,
    """
    request_urn = ('%s/id/%s') % (FreeNASServer.REST_API_VOLUME,
        urllib.parse.quote_plus(self.configuration.ixsystems_dataset_path))
    LOG.debug('_update_volume_stats request_urn : %s', request_urn)
    ret = self.handle.invoke_command(FreeNASServer.SELECT_COMMAND,
                                     request_urn, None)
    LOG.debug("_update_volume_stats response : %s", json.dumps(ret))
    data = {}
    data["volume_backend_name"] = self.backend_name
    data["vendor_name"] =  self.vendor_name
    data["driver_version"] = self.VERSION
    data["storage_protocol"] = self.storage_protocol
    data['total_capacity_gb'] = ix_utils.get_size_in_gb(json.loads(ret['response'])['available']['parsed'] \
        + json.loads(ret['response'])['used']['parsed'])
    data['free_capacity_gb'] = ix_utils.get_size_in_gb(json.loads(ret['response'])['available']['parsed'])
    data['reserved_percentage'] = \
        self.configuration.ixsystems_reserved_percentage
    data['reserved_percentage'] = 0
    data['QoS_support'] = False

    self.stats = data
    return self.stats

[shosti]

I think it’s best if someone forks my code and opens up a new PR—I’m not really comfortable opening up a PR that I have no way of testing.

[marc5feb]

if needed i can build a test setup later to test things out.