[gpfsug-discuss] Spectrum Scale Encryption

Felipe Knop knop at us.ibm.com
Fri Apr 7 15:00:09 BST 2017


All,

A few comments on the topics raised below.

1) All nodes that mount an encrypted file system, and also the nodes with 
management roles on the file system will need access to the keys have the 
proper setup (RKM.conf, etc).

Edward is correct that there was some change in behavior, introduced in 
4.2.1 . Before the change, a mount would fail unless RKM.conf is present 
on the node. In addition, once a policy with encryption rules was applied, 
nodes without the proper encryption setup would unmount the file system. 
With the change, the error gets delayed to when encrypted files are 
accessed.

The change in behavior was introduced based on feedback that unmounting 
the file system in that case was too drastic in that scenario.

>> So on a node which I don't want to have access to any encrypted files, 
do
I just need to have an empty RKM.conf file?

All nodes which mount an encrypted file system should have proper setup 
for encryption, even for a node from where only unencrypted files are 
being accessed.


2)
>> Encrypted file systems do not store data in large MD blocks.  Makes 
sense. This means large MD blocks aren't as useful as they are in 
unencrypted FS, if you are using this. 

Correct. Data is not stored in the inode for encrypted files. On the other 
hand, since encryption metadata is stored as an extended attribute in the 
inode, 4K inodes are still recommended -- especially in cases where a more 
complicated encryption policy is used.

3)
>> Having at least one backup SKLM server is a good idea. 
"kmipServerUri[N+1]" in the conf.

While the documentation claims the FS can continue operation once it 
caches the MEK if an SKLM server goes away, in operation this does NOT 
work as you may expect.  Your users still need access to the FEKs for the 
files your clients work on.  Logs will fill with Key <key> could not be 
fetched. errors. 

Using a backup key server is strongly recommended.

While it's true that the files may still be accessed for a while if the 
key server becomes unreachable, this was not something to be counted on. 
First because keys (MEK) may expire at any time, requiring the key to be 
retrieved from the key server again. Second because a file may require a 
key may be needed that has not been cached before.


4)
>> Secondly ... (and maybe I'm misunderstanding the docs here)

For the Policy
https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.2/com.ibm.spectru

m.scale.v4r22.doc/bl1adv_encryptionpolicyrules.htm

KEYS ('Keyname'[, 'Keyname', ... ])

KeyId:RkmId

RkmId should match the stanza name in RKM.conf?

Correct.

>> If so, it would be useful if the docs used the same names in the 
examples
(RKMKMIP3 vs rkmname3)

And KeyId should match a "Key UUID" in SKLM?

Correct.

We'll review the documentation to ensure that the meaning of the RkmId in 
the examples is clear.


5)

>> Third. My understanding from talking to various IBM people is that we 
need
ISKLM entitlements for NSD Servers, Protocol nodes and AFM gateways
(probably), do we have to do any kind of node registration in ISKLM? Or is
this purely based on the certificates being distributed to clients and
keys are mapped in ISKLM to the client cert to determine if the node is
able to request the key?

I'll work on getting clarifications from the ISKLM folks on this aspect.

  Felipe

----
Felipe Knop                                     knop at us.ibm.com
GPFS Development and Security
IBM Systems
IBM Building 008
2455 South Rd, Poughkeepsie, NY 12601
(845) 433-9314  T/L 293-9314





From:   "Wahl, Edward" <ewahl at osc.edu>
To:     gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date:   04/06/2017 10:55 AM
Subject:        Re: [gpfsug-discuss] Spectrum Scale Encryption
Sent by:        gpfsug-discuss-bounces at spectrumscale.org



This is rather dependant on SS version. 

So what used to happen before 4.2.2.* is that a client would be unable to 
mount the filesystem in question and would give an error in the 
mmfs.log.latest for an SGPanic,   In 4.2.2.* It appears it will now mount 
the file system and then give errors on file access instead.  (just tested 
this on 4.2.2.3) I'll have to read through the changelogs looking for this 
one. 

Depending on your policy for encryption then, this might be exactly what 
you want,  but I REALLY REALLY dislike this behaviour.

To me this means clients can now mount an encrypted FS now and then fail 
during operation.  If I get a client node that comes up improperly, user 
work will start, and it will fail with "Operation not permitted" errors on 
file access.  I imagine my batch system could run through a massive amount 
of jobs on a bad client without anyone noticing immeadiately.  Yet another 
thing we now have to monitor now I guess.  *shrug*

A couple other gotcha's we've seen with Encryption:

Encrypted file systems do not store data in large MD blocks.  Makes sense. 
This means large MD blocks aren't as useful as they are in unencrypted FS, 
if you are using this. 

Having at least one backup SKLM server is a good idea. 
"kmipServerUri[N+1]" in the conf.

While the documentation claims the FS can continue operation once it 
caches the MEK if an SKLM server goes away, in operation this does NOT 
work as you may expect.  Your users still need access to the FEKs for the 
files your clients work on.  Logs will fill with Key <key> could not be 
fetched. errors. 

Ed Wahl
OSC

________________________________________
From: gpfsug-discuss-bounces at spectrumscale.org 
[gpfsug-discuss-bounces at spectrumscale.org] on behalf of Simon Thompson 
(Research Computing - IT Services) [S.J.Thompson at bham.ac.uk]
Sent: Thursday, April 06, 2017 4:20 AM
To: gpfsug-discuss at spectrumscale.org
Subject: [gpfsug-discuss] Spectrum Scale Encryption

We are currently looking at adding encryption to our deployment for some
of our data sets and for some of our nodes. Apologies in advance if some
of this is a bit vague, we're not yet at the point where we can test this
stuff out, so maybe some of it will become clear when we try it out.


For a node that we don't want to have access to any encrypted data, what
do we need to set up?

According to the docs:
https://www.ibm.com/support/knowledgecenter/STXKQY_4.2.2/com.ibm.spectrum.s

cale.v4r22.doc/bl1adv_encryption_prep.htm


"After the file system is configured with encryption policy rules, the
file system is considered encrypted. From that point on, each node that
has access to that file system must have an RKM.conf file present.
Otherwise, the file system might not be mounted or might become 
unmounted."

So on a node which I don't want to have access to any encrypted files, do
I just need to have an empty RKM.conf file?

(If this is the case, would be good to have this added to the docs)


Secondly ... (and maybe I'm misunderstanding the docs here)

For the Policy
https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.2/com.ibm.spectru

m.scale.v4r22.doc/bl1adv_encryptionpolicyrules.htm


KEYS ('Keyname'[, 'Keyname', ... ])


KeyId:RkmId


RkmId should match the stanza name in RKM.conf?

If so, it would be useful if the docs used the same names in the examples
(RKMKMIP3 vs rkmname3)

And KeyId should match a "Key UUID" in SKLM?


Third. My understanding from talking to various IBM people is that we need
ISKLM entitlements for NSD Servers, Protocol nodes and AFM gateways
(probably), do we have to do any kind of node registration in ISKLM? Or is
this purely based on the certificates being distributed to clients and
keys are mapped in ISKLM to the client cert to determine if the node is
able to request the key?

Thanks

Simon

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20170407/fb35fa44/attachment.htm>


More information about the gpfsug-discuss mailing list