Index: nfsv41_middle_op_backchannel_ctl.xml =================================================================== RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_backchannel_ctl.xml,v retrieving revision 1.13 diff -u -r1.13 nfsv41_middle_op_backchannel_ctl.xml --- nfsv41_middle_op_backchannel_ctl.xml 8 Nov 2007 04:15:21 -0000 1.13 +++ nfsv41_middle_op_backchannel_ctl.xml 1 May 2008 04:07:17 -0000 @@ -23,10 +23,10 @@ The arguments of the BACKCHANNEL_CTL call are a subset of the CREATE_SESSION parameters. - In the arguments to BACKCHANNEL_CTL, the + In the arguments of BACKCHANNEL_CTL, the bca_cb_program field and bca_sec_parms fields correspond respectively to the csa_cb_program and - csa_sec_parms of the arguments to CREATE_SESSION + csa_sec_parms fields of the arguments of CREATE_SESSION (). @@ -38,7 +38,7 @@ If the RPCSEC_GSS handle identified by gcbp_handle_from_server does not exist on the server, - the server will return NFS4ERR_NOENT. + the server MUST return NFS4ERR_NOENT. Index: nfsv41_middle_op_bind_conn_to_session.xml =================================================================== RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_bind_conn_to_session.xml,v retrieving revision 1.26 diff -u -r1.26 nfsv41_middle_op_bind_conn_to_session.xml --- nfsv41_middle_op_bind_conn_to_session.xml 25 Feb 2008 20:51:37 -0000 1.26 +++ nfsv41_middle_op_bind_conn_to_session.xml 1 May 2008 04:07:17 -0000 @@ -18,21 +18,21 @@ be the only operation in the COMPOUND procedure. If SP4_NONE () state protection is used, any principal, - security flavor, or RPCSEC_GSS context can invoke the operation. - If SP4_MACH_CRED is used, RPCSEC_GSS must be used with the + security flavor, or RPCSEC_GSS context MAY be used to invoke the operation. + If SP4_MACH_CRED is used, RPCSEC_GSS MUST be used with the integrity or privacy services, using the principal that created the client ID. If SP4_SSV is used, RPCSEC_GSS with the SSV GSS mechanism () and integrity or privacy MUST be used. - If when the client ID was created, the client opted for SP4_NONE + If, when the client ID was created, the client opted for SP4_NONE state protection, the client is not required to use BIND_CONN_TO_SESSION to associate the connection with the session, unless the client wishes to associate the connection with the backchannel. When SP4_NONE protection is used, simply sending a COMPOUND - with a SEQUENCE operation that is sufficient to associate the + request with a SEQUENCE operation is sufficient to associate the connnection with the session specified in SEQUENCE. @@ -40,12 +40,12 @@ wants to associate the connection with the fore channel or the backchannel or both channels. The value CDFC4_FORE_OR_BOTH indicates the client wants to - associate with both the fore channel and backchannel, + associate the connection with both the fore channel and backchannel, but will accept the connection being associated to just the fore channel. The value CDFC4_BACK_OR_BOTH indicates the client wants to associate with both the fore and backchannel, but will accept the - connection being associated with the backchannel. + connection being associated with just the backchannel. The server replies in bctsr_dir which channel(s) the connection is associated with. If the client specified CDFC4_FORE, the server @@ -66,14 +66,19 @@ Invoking BIND_CONN_TO_SESSION on a connection already associated - with the specified session has no effect, and the server SHOULD - respond with NFS4_OK. + with the specified session has no effect, and the server MUST + respond with NFS4_OK, unless the client is demanding changes + to the set of channels the connection is associated with. If + so, the server MUST return NFS4ERR_INVAL. +
- If a session's channel loses all connections, the client needs to use + If a session's channel loses all connections, depending on + the client ID's state protection and type of channel, + the client might need to use BIND_CONN_TO_SESSION to associate a new connection. If the server restarted and does not keep the reply cache in stable storage, the server will not recognize the sessionid. @@ -81,12 +86,15 @@ create a new client ID and session. - Assuming SP4_SSV state protection is being used, - there is an issue if SET_SSV is sent, no response + Suppose SP4_SSV state protection is being used, + and BIND_CONN_TO_SESSION is among the operations + included in the spo_must_enforce set when the + client ID was created (). + If so, there is an issue if SET_SSV is sent, no response is returned, and the last connection associated - with the client ID disconnects. The client, per - the sessions model, needs to retry the SET_SSV. But - it needs a new connection to do so, and needs to + with the client ID drops. The client, per + the sessions model, MUST retry the SET_SSV. But + it needs a new connection to do so, and MUST associate that connection with the session via a BIND_CONN_TO_SESSION authenticated with the SSV GSS mechanism. The problem is that the RPCSEC_GSS @@ -94,21 +102,41 @@ key and the SSV may have changed. While there are multiple recovery strategies, a single, general strategy - is described here. First the client reconnects. - The client assumes the SET_SSV was executed, - and so sends BIND_CONN_TO_SESSION with the subkey derived from - new SSV used as key for the message integrity code in - the RPCSEC_GSS credential message integrity codes. - If the server returns an RPC authentication error, - this means the server's current SSV was not changed, - and the SET_SSV was not executed. The client then - tries BIND_CONN_TO_SESSION with the subkey derived from the - old SSV as the - key for the RPCSEC_GSS message integrity code. This - should not return an RPC authentication error. - If it does, an implementation error has occurred on - either the client or server, and the client has to - create a new client ID. + is described here. + + + The client reconnects. + + + The client assumes the SET_SSV was executed, + and so sends BIND_CONN_TO_SESSION with the subkey derived from + new SSV (what SET_SSV would have set the SSV to) + used as the key for the RPCSEC_GSS credential message integrity codes. + + + If the request succeeds, this means the original attempted SET_SSV + did execute successfully. The client re-sends the original + SET_SSV, which the server will reply from via the + the reply cache. + + + If the server returns an RPC authentication error, + this means the server's current SSV was not changed, + (and the SET_SSV was likely not executed). The client then + tries BIND_CONN_TO_SESSION with the subkey derived from the + old SSV as the + key for the RPCSEC_GSS message integrity codes. + + + The attempted BIND_CONN_TO_SESSION with the old SSV + should succeed. If so the client re-sends the original + SET_SSV. If the original SET_SSV was not executed, then the + server executes it. If the original SET_SSV was executed, but + failed, the server will return the SET_SSV from the reply + cache. + + +
Index: nfsv41_middle_op_cb_illegal.xml =================================================================== RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_cb_illegal.xml,v retrieving revision 1.5 diff -u -r1.5 nfsv41_middle_op_cb_illegal.xml --- nfsv41_middle_op_cb_illegal.xml 8 Nov 2007 04:15:21 -0000 1.5 +++ nfsv41_middle_op_cb_illegal.xml 1 May 2008 04:07:17 -0000 @@ -15,10 +15,12 @@
- This operation is a placeholder for encoding a result to handle the - case of the client sending an operation code within COMPOUND that - is not supported. See the COMPOUND procedure description for more - details. + This operation is a placeholder for encoding a + result to handle the case of the client sending + an operation code within COMPOUND that is not + defined in the NFSv4.1 specification. See for more details. + The status field of CB_ILLEGAL4res MUST be set to Index: nfsv41_middle_op_create_session.xml =================================================================== RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_create_session.xml,v retrieving revision 1.46 diff -u -r1.46 nfsv41_middle_op_create_session.xml --- nfsv41_middle_op_create_session.xml 20 Apr 2008 16:28:33 -0000 1.46 +++ nfsv41_middle_op_create_session.xml 1 May 2008 04:07:18 -0000 @@ -52,14 +52,14 @@ This is the client ID the new session will be associated - with. The corresponding result is csr_sessionid, sessionid + with. The corresponding result is csr_sessionid, the sessionid of the new session. Each client ID serializes CREATE_SESSION via a per client ID - sequence number. See - . + sequence number (see + ). The corresponding result is csr_sequence, which MUST be equal to csa_sequence. @@ -72,7 +72,7 @@ the server accept the value. If it is not acceptable, the server MAY use a different value. Regardless, the server MUST return the value the session will - uses (which will be either what the client offered, or what + use (which will be either what the client offered, or what the server is insisting on). return the value used to the client. These parameters have the following interpretation. @@ -119,7 +119,7 @@ If CREATE_SESSION4_FLAG_CONN_RDMA is set in csa_flags, - the connection CREATE_SESSION is called over + and if the connection CREATE_SESSION is called over is currently in non-RDMA mode, but has the capability to operate in RDMA mode, then client is requesting the server agree to "step up" to RDMA mode @@ -134,10 +134,9 @@ - - + - The csa_fore_char_attrs and csa_back_chan_attrs + The csa_fore_chan_attrs and csa_back_chan_attrs fields apply to attributes of the fore channel (which conveys requests originating from the client to the server), @@ -177,15 +176,15 @@ The maximum size of a COMPOUND or CB_COMPOUND reply that - the replier will - accept from the requester including RPC headers (see + the requester will + accept from the replier including RPC headers (see the ca_maxrequestsize definition). The NFSv4.1 server MUST NOT increase the value of this parameter in the CREATE_SESSION results. However, if the client selects a value for ca_maxresponsesize such that a replier on a channel could never send a response, the server SHOULD return - NFS4ERR_TOOSMALL to in the CREATE_SESSION reply. + NFS4ERR_TOOSMALL in the CREATE_SESSION reply. If a requester sends a request for which the size of the reply would exceed this value, the replier will return NFS4ERR_REP_TOO_BIG, @@ -290,8 +289,9 @@ The first RPCSEC_GSS handle, gcbp_handle_from_server, is the fore handle the server returned to - the client (in the handle field of data type - rpc_gss_init_res) when the RPCSEC_GSS context + the client (either in the handle field of data type + rpc_gss_init_res or one of the elements of the spi_handles + field returned in the reply to EXCHANGE_ID) when the RPCSEC_GSS context was created on the server. The second handle, gcbp_handle_from_client, is the back handle the client will map the RPCSEC_GSS context to. The @@ -303,8 +303,8 @@ rpc_gss_cred_t (see Section 5, "Elements of the RPCSEC_GSS Security Protocol" of ) in callback RPCs. - The server must use the RPCSEC_GSS security service - specified in gcbp_service, i.e. it must set the + The server MUST use the RPCSEC_GSS security service + specified in gcbp_service, i.e. it MUST set the "service" field of the rpc_gss_cred_t data type in RPCSEC_GSS credential to the value of gcbp_service (see Section 5.3.1, "RPC Request Header", of ). @@ -377,20 +377,19 @@ - When a client sends a successful EXCHANGE_ID and it is returned - an unconfirmed client ID, the client is also returned - eir_sequenceid, and the client is expected to set - the value of csa_sequenceid in the client ID confirming - CREATE_SESSION it sends with - that client ID to the value of eir_sequenceid. - After EXCHANGE_ID, the server initializes the client ID - slot to be equal to eir_sequenceid - 1 (accounting - for underflow), and records a contrived CREATE_SESSION - result with a "cached" result of - NFS4ERR_SEQ_MISORDERED. - With the slot thus - initialized, the processing of the CREATE_SESSION - operation is divided into four phases: + When a client sends a successful EXCHANGE_ID and + it is returned an unconfirmed client ID, the client + is also returned eir_sequenceid, and the client is + expected to set the value of csa_sequenceid in the + client ID-confirming-CREATE_SESSION it sends with + that client ID to the value of eir_sequenceid. When + EXCHANGE_ID returns a new, unconfirmed client ID, + the server initializes the client ID slot to be equal + to eir_sequenceid - 1 (accounting for underflow), + and records a contrived CREATE_SESSION result + with a "cached" result of NFS4ERR_SEQ_MISORDERED. + With the slot thus initialized, the processing of the + CREATE_SESSION operation is divided into four phases: @@ -440,16 +439,28 @@ is the expected confirmation of an unconfirmed record. - { *, *, principal_arg, clientid_arg, unconfirmed } + { ownerid, verifier, principal_arg, clientid_arg, unconfirmed } - The record is replaced with: + As noted in , + the server might also have the following confirmed record. - { *, *, principal_arg, clientid_arg, confirmed } + { ownerid, old_verifier, principal_arg, old_clientid, confirmed } + + + The server schedules the replacement of + both records are atomically replaced with: + + + { ownerid, verifier, principal_arg, clientid_arg, confirmed } - The processing of the operation continues to session creation. + The processing of CREATE_SESSION continues on to session creation. + Once the session is successfully created, the scheduled client + record replacement is committed. If the session is not + successfully created, then no changes are made to any client + records on the server. @@ -487,12 +498,13 @@ parameter values used (including whether the CREATE_SESSION4_FLAG_PERSIST flag is set and has been accepted by the server) and allocating space - for the session reply cache. For each slot in the + for the session reply cache (if there is not enough + space, the server returns NFS4ERR_NOSPC). For each slot in the reply cache, the server sets the sequence id to zero (0), and records an entry containing a COMPOUND - reply with a zero operations and the error of + reply with zero operations and the error NFS4ERR_SEQ_MISORDERED. This way, if the first - SEQUENCE request sent has a sequenceid equal to + SEQUENCE request sent has a sequence id equal to zero, the server can simply return what is in the reply cache: NFS4ERR_SEQ_MISORDERED. The client initializes its reply cache for receiving callbacks @@ -512,13 +524,13 @@ If properties of the new connection are different enough that the arguments to CREATE_SESSION must change, then a non-retry MUST be sent. The server will eventually dispose - of any session that was created. + of any session that was created on the original connection. On the backchannel, the client and server might wish to - have many slots, in some cases perhaps more that the fore channel in + have many slots, in some cases perhaps more that the fore channel, in to deal with the situations where the network link has high latency and is the primary bottleneck for response to recalls. If so, and if the @@ -564,9 +576,5 @@ - - If the server cannot reserve space for the reply cache, it - MAY return NFS4ERR_NOSPC. -
Index: nfsv41_middle_op_destroy_clientid.xml =================================================================== RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_destroy_clientid.xml,v retrieving revision 1.8 diff -u -r1.8 nfsv41_middle_op_destroy_clientid.xml --- nfsv41_middle_op_destroy_clientid.xml 20 Nov 2007 06:33:58 -0000 1.8 +++ nfsv41_middle_op_destroy_clientid.xml 1 May 2008 04:07:18 -0000 @@ -17,9 +17,9 @@ The DESTROY_CLIENTID operation destroys the client ID. If there are sessions (both idle and non-idle), opens, locks, delegations, layouts, - and wants () + and/or wants () associated with the unexpired lease of the client - ID the server MUST return NFS4ERR_CLIENTID_BUSY. + ID, the server MUST return NFS4ERR_CLIENTID_BUSY. DESTROY_CLIENTID MAY be preceded with a SEQUENCE operation as long as the client ID derived from the sessionid of SEQUENCE is not the same as the client Index: nfsv41_middle_op_exchange_id.xml =================================================================== RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_exchange_id.xml,v retrieving revision 1.34 diff -u -r1.34 nfsv41_middle_op_exchange_id.xml --- nfsv41_middle_op_exchange_id.xml 31 Mar 2008 23:18:57 -0000 1.34 +++ nfsv41_middle_op_exchange_id.xml 1 May 2008 04:07:18 -0000 @@ -56,10 +56,10 @@ In addition to the client ID and sequence id, the server returns a server owner (eir_server_owner) and - eir_server_scope. The former field is used for + server scope (eir_server_scope). The former field is used for network trunking as described in . The latter field is used to - allow clients to determine when clientids sent by + allow clients to determine when client IDs sent by one server may be recognized by another in the event of file system migration (see ). @@ -77,7 +77,7 @@ are distinct. If they are distinct, then the client will need to note the destination network addresses of the connections used with each server, and use - network address as the final discriminator. + the network address as the final discriminator. The server, as defined by the unique identity expressed @@ -158,7 +158,7 @@ The length of the SSV. This property is - represented by the spi_ssv_len in the EXCHANGE_ID + represented by the spi_ssv_len field in the EXCHANGE_ID results. Once the client ID is confirmed, @@ -186,6 +186,12 @@ The property may be updated by subsequent EXCHANGE_ID requests. + + The server's implementation ID as represented by + the eir_server_impl_id field of the reply. + The property may be updated by replies to subsequent EXCHANGE_ID + requests. + @@ -197,7 +203,7 @@ a bit is set or cleared on the arguments' flags does not force the server to set or clear the same bit on the results' side. Bits not defined above - should not be set in the eia_flags field. If they + cannot be set in the eia_flags field. If they are, the server MUST reject the operation with NFS4ERR_INVAL. @@ -270,10 +276,10 @@ Sub- of in ) will apply. + target="OP_EXCHANGE_ID_IMPLEMENTATION" /> will apply. Note that if the operation succeeds - and returns a client ID that is already + and returns a client ID that was previously confirmed, the server MUST set the EXCHGID4_FLAG_CONFIRMED_R bit in eir_flags. @@ -458,7 +464,8 @@ If spo_must_enforce in the results has BIND_CONN_TO_SESSION set, then connection binding enforcement is enabled, and - the client MUST use the machine or SSV credential on calls + the client MUST use the machine (if SP4_MACH_CRED protection is used) + or SSV (if SP4_SSV protection is used) credential on calls to BIND_CONN_TO_SESSION. @@ -544,12 +551,12 @@ This is the number of SSV versions the client wants - the server to maintain (i.e. each call to SET_SSV + the server to maintain (i.e. each successful call to SET_SSV produces a new version of the SSV). If ssp_window is zero, the server MUST return NFS4ERR_INVAL. The server responds with spi_window, which MUST NOT exceed ssp_window, and MUST be at least one (1). - Any requests on the backchannel or forechannel that + Any requests on the backchannel or fore channel that are using a version of the SSV that is outside the window will fail with an ONC RPC authentication error, and the requester will have to retry them with the same slot id and @@ -863,7 +870,7 @@ If the EXCHGID4_FLAG_UPD_CONFIRMED_REC_A flag is not set, and the server has the following unconfirmed record then the client is attempting EXCHANGE_ID again on an - unconfirmed client ID, perhaps do to a retry, or perhaps + unconfirmed client ID, perhaps due to a retry, or perhaps due to a client restart before client ID confirmation (i.e. before CREATE_SESSION was called), or some other reason. @@ -923,12 +930,47 @@ After processing, clientid_ret is returned to the client and - the client record is replaced with: + this client record is added: { ownerid_arg, verifier_arg, principal_arg, clientid_ret, unconfirmed } + + The previously described confirmed record + continues to exist, and thus the same + ownerid_arg exists in both a confirmed and + unconfirmed state at the same time. The number + of states can collapse to one once the server + receives an applicable CREATE_SESSION or + EXCHANGE_ID. + + + + + If the server subsequently receives a successful + CREATE_SESSION that confirms clientid_ret, + then the server atomically destroys the + confirmed record and makes the unconfirmed + record confirmed as described in . + + + + + If the server instead subsequently receives + an EXCHANGE_ID with the client owner equal + to ownerid_arg, one strategy is to simply + delete the unconfirmed record, and process the + EXCHANGE_ID as described in the entirety of + . + + + + + + Index: nfsv41_middle_op_free_stateid.xml =================================================================== RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_free_stateid.xml,v retrieving revision 1.9 diff -u -r1.9 nfsv41_middle_op_free_stateid.xml --- nfsv41_middle_op_free_stateid.xml 31 Mar 2008 23:18:57 -0000 1.9 +++ nfsv41_middle_op_free_stateid.xml 1 May 2008 04:07:18 -0000 @@ -35,10 +35,5 @@ error. -
- - No discussion at this time. - -
Index: nfsv41_middle_op_get_dir_delegation.xml =================================================================== RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_get_dir_delegation.xml,v retrieving revision 1.10 diff -u -r1.10 nfsv41_middle_op_get_dir_delegation.xml --- nfsv41_middle_op_get_dir_delegation.xml 10 Nov 2007 04:53:25 -0000 1.10 +++ nfsv41_middle_op_get_dir_delegation.xml 1 May 2008 04:07:18 -0000 @@ -36,12 +36,12 @@ delegation is returned by calling the DELEGRETURN operation. - The server may not be able to support notifications of certain + The server might not be able to support notifications of certain events. If the client asks for such notifications, the server - must inform the client of its inability to do so as part of the + MUST inform the client of its inability to do so as part of the GET_DIR_DELEGATION reply by not setting the appropriate bits in the supported notifications bitmask, gddr_notification, contained - in the reply. The server should not add bits to gddr_notification + in the reply. The server MUST NOT add bits to gddr_notification that the client did not request. @@ -59,19 +59,19 @@ operation to indicate that a directory delegation is available. If the server does not wish to honor the "want" or is not able to do so, it returns the error NFS4ERR_DIRDELEG_UNAVAIL. If the - delegation is immediately available, the server may return it with + delegation is immediately available, the server SHOULD return it with the response to the operation, rather than via a callback.
- Directory delegation provides the benefit of improving cache + Directory delegations provide the benefit of improving cache consistency of namespace information. This is done through synchronous callbacks. A server must support synchronous callbacks in order to support directory delegations. In addition to that, asynchronous notifications provide a way to reduce network traffic as well as improve client performance in certain - conditions. Notifications would not be requested when the goal + conditions. Notifications should not be requested when the goal is just cache consistency. @@ -105,8 +105,8 @@ all changes that would otherwise invalidate its name cache. The kind of notification a client asks for may depend on the directory size, its rate of change and the applications being - used to access that directory. However, the conditions under - which a client might ask for a notification, is out of the scope + used to access that directory. The enumeration of the conditions under + which a client might ask for a notification is out of the scope of this specification. @@ -114,7 +114,7 @@ will set bits in the gdda_dir_attributes bitmap to indicate which attributes it wants to be notified of. If the server does not support - notifications for changes to a certain attribute, it should not + notifications for changes to a certain attribute, it SHOULD NOT set that attribute in the supported attribute bitmap specified in the reply (gddr_dir_attributes). The client will also set in the gdda_child_attributes bitmap the attributes @@ -131,8 +131,8 @@ delay factor is set to zero, that indicates to the server that the client wants to be notified of any attribute changes as soon as they occur. If the delay factor is set to N seconds, the server will - make a best effort guarantee that attribute updates are not out - of sync by more than N seconds. + make a best effort guarantee that attribute updates are + synchronized within N seconds. If the client asks for a delay factor that the server does not support or that may cause significant resource consumption on the server by causing @@ -142,10 +142,10 @@ gddr_child_attributes and gddr_dir_attributes bitmaps in the response. - The client should use a security flavor that the file system is - exported with. If it uses a different flavor, the server should + The client should use a security tuple that the file system is + exported with. If it uses a different tuple, the server should return NFS4ERR_WRONGSEC - to the operation that precedes GET_DIR_DELEGATION + to the operation that both precedes GET_DIR_DELEGATION and sets the current filehandle. Index: nfsv41_middle_op_getdeviceinfo.xml =================================================================== RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_getdeviceinfo.xml,v retrieving revision 1.19 diff -u -r1.19 nfsv41_middle_op_getdeviceinfo.xml --- nfsv41_middle_op_getdeviceinfo.xml 13 Feb 2008 16:17:21 -0000 1.19 +++ nfsv41_middle_op_getdeviceinfo.xml 1 May 2008 04:07:18 -0000 @@ -11,15 +11,15 @@
- Returns device address information for the specified device ID. + Returns pNFS storage device address information for the specified device ID. The client identifies the device information to be returned by providing the gdia_device_id and gdia_layout_type that uniquely - identify the device address. The client provides gdia_maxcount + identify the device. The client provides gdia_maxcount to limit the number of bytes for the result. This maximum size represents all of the data being returned within the GETDEVICEINFO4resok structure and includes the XDR overhead. The server may return less data. If the server is unable to - return the information within the gdia_maxcount limit, the error + return any information within the gdia_maxcount limit, the error NFS4ERR_TOOSMALL will be returned. However, if gdia_maxcount is zero, NFS4ERR_TOOSMALL MUST NOT be returned. @@ -60,7 +60,7 @@ If the client wants to just update or turn off notifications, it MAY issue GETDEVICEINFO with gdia_maxcount set to zero. - In that event, if the device ID is valid, the da_addr_body + In that event, if the device ID is valid, the reply's da_addr_body field of the gdir_device_addr field will be of zero length. @@ -119,8 +119,8 @@ one layout has not been revoked, the client should send a GETDEVICEINFO on the device ID to verify that the device ID has been deleted. If GETDEVICEINFO indicates the device ID - does not exist, the client then assumes the server is broken, - and recovers issuing EXCHANGE_ID. + does not exist, the client then assumes the server is faulty, + and recovers issuing by EXCHANGE_ID. If the client does not have layouts that refer to the device ID, no harm is done. @@ -136,7 +136,7 @@ addressing mappings have changed. The client should assume that the results from the in progress GETDEVICEINFO will be stale for the device ID - once received, and so it should send a GETDEVICEINFO + once received, and so it should send another GETDEVICEINFO on the device ID. Index: nfsv41_middle_op_getdevicelist.xml =================================================================== RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_getdevicelist.xml,v retrieving revision 1.16 diff -u -r1.16 nfsv41_middle_op_getdevicelist.xml --- nfsv41_middle_op_getdevicelist.xml 29 Jan 2008 08:19:32 -0000 1.16 +++ nfsv41_middle_op_getdevicelist.xml 1 May 2008 04:07:18 -0000 @@ -19,14 +19,18 @@ belongs to the file system (i.e. all file objects sharing the same fsid as that of the current filehandle), and the layout type in gdia_layout_type. Since - this operation may require multiple calls to enumerate all the + this operation might require multiple calls to enumerate all the device IDs (and is thus similar to the READDIR operation), the client also provides gdia_cookie and gdia_cookieverf to specify the current cursor position in the - list. + list. When the client wants to read from the beginning of the + file system's device mappings, it sets gdla_cookie to zero. The + field gdla_cookieverf MUST be ignored by the server when + gdla_cookie is zero. The client provides gdla_maxdevices to limit the number of device IDs - in the result. + in the result. If gdla_maxdevices is zero, the server MUST return + NFS4ERR_INVAL. The server MAY return fewer device IDs. Index: nfsv41_middle_op_layoutcommit.xml =================================================================== RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_layoutcommit.xml,v retrieving revision 1.22 diff -u -r1.22 nfsv41_middle_op_layoutcommit.xml --- nfsv41_middle_op_layoutcommit.xml 25 Feb 2008 20:51:37 -0000 1.22 +++ nfsv41_middle_op_layoutcommit.xml 1 May 2008 04:07:18 -0000 @@ -15,13 +15,13 @@ filehandle, client ID (derived from the sessionid in the preceding SEQUENCE operation), byte range, and stateid. Since layouts are sub-dividable, a smaller portion of a layout, - retrieved via LAYOUTGET, may be committed. The region being + retrieved via LAYOUTGET, can be committed. The region being committed is specified through the byte range (loca_offset and loca_length). This region MUST overlap with one or more existing layouts previously granted via LAYOUTGET (), each with an iomode of LAYOUTIOMODE4_RW. In the - case where any held layout segments iomode is not - LAYOUTIOMODE4_RW the server should return the error + case where the iomode of any held layout segment is not + LAYOUTIOMODE4_RW, the server should return the error NFS4ERR_BAD_IOMODE. For the case where the client does not hold matching layout segment(s) for the defined region, the server should return the error @@ -73,9 +73,9 @@ "reclaim" field to FALSE. - The loca_stateid is a layout stateid value as returned by - previously successful layout operations ( - see ). + The loca_stateid is a layout stateid value as + returned by previously successful layout operations + (see ). @@ -99,7 +99,7 @@ time. If the metadata server uses the client provided modification time, it should ensure time does not flow backwards. If the client wants to force the metadata server to set an exact time, - the client should use a SETATTR operation in a compound right + the client should use a SETATTR operation in a COMPOUND right after LAYOUTCOMMIT. See for more details. If the client desires the resultant modification time it should construct the COMPOUND so that a GETATTR Index: nfsv41_middle_op_layoutget.xml =================================================================== RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_layoutget.xml,v retrieving revision 1.26 diff -u -r1.26 nfsv41_middle_op_layoutget.xml --- nfsv41_middle_op_layoutget.xml 31 Mar 2008 23:18:57 -0000 1.26 +++ nfsv41_middle_op_layoutget.xml 1 May 2008 04:07:18 -0000 @@ -12,12 +12,12 @@
Requests a layout from the metadata server for reading or - writing (and reading) the file given by the filehandle at the + writing the file given by the filehandle at the byte range specified by offset and length. Layouts are identified by the client ID (derived from the sessionid in the preceding SEQUENCE operation), current filehandle, layout type (loga_layout_type), and the layout stateid (loga_stateid). The - use of the loga_iomode depends upon the layout type, but should + use of the loga_iomode field depends upon the layout type, but should reflect the client's data access intent. @@ -27,14 +27,16 @@ - The LAYOUTGET operation returns layout information for the specified - byte range: a layout. To get a layout from a - specific offset through the end-of-file, regardless of the file's - length, a loga_length field with all bits set to 1 (one) should be used. - If loga_length is zero, or if a loga_length which is not all bits set to - one is specified, and loga_length when added to loga_offset exceeds the - maximum 64-bit unsigned integer value, the error NFS4ERR_INVAL will - result. + The LAYOUTGET operation returns layout information + for the specified byte range: a layout. To get + a layout from a specific offset through the + end-of-file, regardless of the file's length, a + loga_length field set to NFS4_UINT64_MAX is used. + If loga_length is zero, or if a loga_length which is + not NFS4_UINT64_MAX is specified, and the sum of loga_length + and loga_offset exceeds NFS4_UINT64_MAX, + the error NFS4ERR_INVAL will result. + The loga_minlength field specifies the minimum length of @@ -138,7 +140,7 @@ step from the client's and server's interaction. - The logr_stateid, as with all stateid processing, is returned to + The logr_stateid stateid is returned to the client for use in subsequent layout related operations. See , , and for a further @@ -200,12 +202,18 @@
- Typically, LAYOUTGET will be called as part of a compound RPC after - an OPEN operation and results in the client having location - information for the file; a client may also hold a layout across - multiple OPENs. The client specifies a layout type that limits what - kind of layout the server will return. This prevents servers from + Typically, LAYOUTGET will be called as part of a + COMPOUND request after an OPEN operation and results + in the client having location information for the + file; this requires that loga_stateid be set to the + special stateid that tells the server to use the + current stateid, which is set by OPEN (see ) . A client may also hold + a layout across multiple OPENs. The client specifies + a layout type that limits what kind of layout the + server will return. This prevents servers from issuing layouts that are unusable by the client. + Once the client has obtained a layout referring to a particular Index: nfsv41_middle_op_layoutreturn.xml =================================================================== RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_layoutreturn.xml,v retrieving revision 1.28 diff -u -r1.28 nfsv41_middle_op_layoutreturn.xml --- nfsv41_middle_op_layoutreturn.xml 27 Mar 2008 21:05:40 -0000 1.28 +++ nfsv41_middle_op_layoutreturn.xml 1 May 2008 04:07:18 -0000 @@ -13,23 +13,27 @@
- This operation returns one or more layouts represented by the - client ID (derived from the sessionid in the preceding SEQUENCE - operation), lora_layout_type, and lora_iomode. When - lr_returntype is LAYOUTRETURN4_FILE, the returned layout is - further identified by the current filehandle, lrf_offset, - lrf_length, and lrf_stateid. If the - lrf_length is all 1s, all bytes of the layout, starting at - lrf_offset are returned. - When lr_returntype is - LAYOUTRETURN4_FSID the current filehandle is used to identify - the file system and all layouts matching the client ID, - lora_layout_type, and lora_iomode are returned. When - lr_returntype is LAYOUTRETURN4_ALL all layouts matching the - client ID, lora_layout_type, and lora_iomode are returned and - the current filehandle is not used. After this call, the client - MUST NOT use the returned layout(s) and the associated storage - protocol to access the file data. + This operation returns from the client to the server + one or more layouts represented by the client ID + (derived from the sessionid in the preceding SEQUENCE + operation), lora_layout_type, and lora_iomode. + When lr_returntype is LAYOUTRETURN4_FILE, the + returned layout is further identified by the current + filehandle, lrf_offset, lrf_length, and lrf_stateid. + If the lrf_length field is NFS4_UINT64_MAX, all bytes + of the layout, starting at lrf_offset are returned. + When lr_returntype is LAYOUTRETURN4_FSID, the + current filehandle is used to identify the file + system and all layouts matching the client ID, + the fsid of the file system, lora_layout_type, and + lora_iomode are returned. When lr_returntype is + LAYOUTRETURN4_ALL, all layouts matching the client + ID, lora_layout_type, and lora_iomode are returned + and the current filehandle is not used. After this + call, the client MUST NOT use the returned layout(s) + and the associated storage protocol to access the + file data. + If the set of layouts designated in the case of @@ -76,19 +80,22 @@ Return of a layout or all layouts does not invalidate the mapping of storage device ID to storage device address which - remains in effect until specifically recalled or changed via - notification callbacks. + remains in effect until specifically changed or deleted via + device ID notification callbacks. - The lora_reclaim field set to TRUE in a LAYOUTRETURN request - specifies that the client is attempting to return a layout that - was acquired before the restart of the metadata server during the - metadata server's grace period. When returning layouts that were - acquired during the metadata server's grace period MUST set the - lora_reclaim field to FALSE. The lora_reclaim field MUST be set to - FALSE also when lr_layoutreturn is LAYOUTRETURN4_FSID or - LAYOUTRETURN4_ALL. See LAYOUTCOMMIT for more details. + If the lora_reclaim field is set to TRUE, the + client is attempting to return a layout that + was acquired before the restart of the metadata + server during the metadata server's grace period. + When returning layouts that were acquired during + the metadata server's grace period, the client MUST set the + lora_reclaim field to FALSE. The lora_reclaim field + MUST be set to FALSE also when lr_layoutreturn is + LAYOUTRETURN4_FSID or LAYOUTRETURN4_ALL. See LAYOUTCOMMIT for + more details. + Layouts may be returned when recalled or voluntarily (i.e., @@ -98,26 +105,30 @@ returning the layout. - If the client is returning the layout in response to a - CB_LAYOUTRECALL where the lor_recalltype was LAYOUTRECALL4_FILE, the - client should use the lor_stateid value from CB_LAYOUTRECALL - as the value for lrf_stateid. Otherwise, it should use - logr_stateid (from a previous LAYOUTGET result) or - lorr_stateid (from a previous LAYRETURN result). - This is done to indicate the point - in time (in terms of layout stateid transitions) when the recall - was sent. The client must use the precise lora_recallstateid - value and not set the seqid to zero. Otherwise NFS4ERR_BAD_STATEID - will be returned. NFS4ERR_OLD_STATEID can be returned if the - client is using an old seqid, and the server knows the - client should not be using the old seqid. E.g. the client uses - the seqid on slot 1 of the session, received the response with - the new seqid, and uses the slot to send another request with the - old seqid. + If the client returns the layout in response to a + CB_LAYOUTRECALL where the lor_recalltype field of the + clora_recall field was LAYOUTRECALL4_FILE, the client + should use the lor_stateid value from CB_LAYOUTRECALL + as the value for lrf_stateid. Otherwise, it should + use logr_stateid (from a previous LAYOUTGET result) + or lorr_stateid (from a previous LAYRETURN result). + This is done to indicate the point in time (in terms + of layout stateid transitions) when the recall was + sent. The client uses the precise lora_recallstateid + value and MUST NOT set the stateid's seqid to + zero; otherwise NFS4ERR_BAD_STATEID MUST be + returned. NFS4ERR_OLD_STATEID can be returned if + the client is using an old seqid, and the server + knows the client should not be using the old + seqid. E.g. the client uses the seqid on slot 1 of + the session, received the response with the new + seqid, and uses the slot to send another request + with the old seqid. + If a client fails to return a layout - in a timely manner, then the metadata server should use its + in a timely manner, then the metadata server SHOULD use its control protocol with the storage devices to fence the client from accessing the data referenced by the layout. See for more details. @@ -132,27 +143,34 @@ NFS4ERR_INVAL is returned. - If the operation specified lr_returntype of LAYOUTRETURN4_FILE, - then lrs_stateid will represent the layout stateid as + If the operation set the lr_returntype field to LAYOUTRETURN4_FILE, + then the lrs_stateid field will represent the layout stateid as updated for this operation's processing; the current stateid will also be updated to match the returned value. If the last byte of any layout for the current file, client ID, and layout type is being returned and there are no remaining pending CB_LAYOUTRECALL operations for which a LAYOUTRETURN - operation must be done as a completing operation, lrs_present - MUST be FALSE, and thus no stateid will be returned. + operation must be done, lrs_present + MUST be FALSE, and thus no stateid will be returned and the current + stateid will be cleared. On success, the current filehandle retains its value. - The server MAY require that the principal, security flavor, and - if applicable, the GSS mechanism, combination that acquired the - layout also be the one to send LAYOUTRETURN. This might not be - possible if credentials for the principal are no longer - available. The server MAY allow the machine credential or SSV - credential (see ) to send - LAYOUTRETURN. + If the EXCHGID4_FLAG_BIND_PRINC_STATEID + capability is set on the client ID (see ), the server will + require that the principal, security flavor, + and if applicable, the GSS mechanism, combination + that acquired the layout also be the one to send + LAYOUTRETURN. This might not be possible + if credentials for the principal are no + longer available. The server will allow the + machine credential or SSV credential (see ) to send LAYOUTRETURN + if LAYOUTRETURN's operation code was set in the + spo_must_allow result of EXCHANGE_ID.
@@ -164,8 +182,7 @@ client is returning the layout for some recalled range the server may recall a superset of that range (e.g. LAYOUTRECALL4_ALL); the final return operation for the latter must block until the former layout - recall is done - when its corresponding final return operation is - replied. + recall is done.
Returning all layouts in a file system using LAYOUTRETURN4_FSID is Index: nfsv41_middle_op_reclaim_complete.xml =================================================================== RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_reclaim_complete.xml,v retrieving revision 1.13 diff -u -r1.13 nfsv41_middle_op_reclaim_complete.xml --- nfsv41_middle_op_reclaim_complete.xml 25 Feb 2008 20:51:37 -0000 1.13 +++ nfsv41_middle_op_reclaim_complete.xml 1 May 2008 04:07:18 -0000 @@ -33,7 +33,7 @@ is being done. This indicates that recovery of locks for a single fs (the one designated by the current filehandle) due to a file system transition have been completed. Presence - of a current filehandle is only required when rca_one_fs is true. + of a current filehandle is only required when rca_one_fs is set to TRUE. @@ -43,7 +43,7 @@ completed recovery. Once the client sends RECLAIM_COMPLETE, the server will not allow the client to do subsequent reclaims of locking state for that scope - and will return NFS4ERR_NO_GRACE, if these are attempted. + and if these are attempted, will return NFS4ERR_NO_GRACE. Whenever a client establishes a new client ID and before it does @@ -51,13 +51,13 @@ global RECLAIM_COMPLETE, even if there are no locks to reclaim. If non-reclaim locking operations are done before the RECLAIM_COMPLETE, a NFS4ERR_GRACE - will be returned. + error will be returned. Similarly, when the client accesses a file system on a new server, before it sends the first non-reclaim operation that obtains a lock on this new server, it must do a RECLAIM_COMPLETE - with rca_one_fs true and current filehandle within that file system, + with rca_one_fs set to TRUE and current filehandle within that file system, even if there are no locks to reclaim. If non-reclaim locking operations are done on that file system before the RECLAIM_COMPLETE, a NFS4ERR_GRACE will be returned. @@ -76,7 +76,7 @@ discussion of edge conditions related to lock reclaim. - Once the client has done a RECLAIM_COMPLETE, it indicates readiness + By sending a RECLAIM_COMPLETE, the client indicates readiness to proceed to do normal non-reclaim locking operations. The client should be aware that such operations may temporarily result in NFS4ERR_GRACE errors until the server is ready to terminate its @@ -87,8 +87,9 @@ Servers will typically use the information as to when reclaim activity is complete to reduce the length of the grace period. - When the server maintains a list of clients that may have locks - in persistent storage, it is in a position to use the fact that + When the server maintains in persistent storage + a list of clients that might have had locks, + it is in a position to use the fact that all such clients have done a RECLAIM_COMPLETE to terminate the grace period and begin normal operations (i.e. grant requests for new locks) sooner than it might otherwise. @@ -103,7 +104,7 @@ RECLAIM_COMPLETE should only be done once for each server instance, or occasion of the transition of a file system. - If it is done a second time, an NFS4ERR_COMPLETE_ALREADY will + If it is done a second time, the error NFS4ERR_COMPLETE_ALREADY will result. Note that because of the session feature's retry protection, retries of COMPOUND requests containing RECLAIM_COMPLETE operation will not result Index: nfsv41_middle_op_sequence.xml =================================================================== RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_sequence.xml,v retrieving revision 1.39 diff -u -r1.39 nfsv41_middle_op_sequence.xml --- nfsv41_middle_op_sequence.xml 31 Mar 2008 23:18:57 -0000 1.39 +++ nfsv41_middle_op_sequence.xml 1 May 2008 04:07:18 -0000 @@ -24,7 +24,7 @@ returned when it is found in any position in a COMPOUND beyond the first. Operations other than SEQUENCE, BIND_CONN_TO_SESSION, EXCHANGE_ID, CREATE_SESSION, and DESTROY_SESSION, - may not appear as the first operation in a + MUST NOT appear as the first operation in a COMPOUND. Such operations MUST yield the error NFS4ERR_OP_NOT_IN_SESSION if they do appear at the start of a COMPOUND. @@ -221,13 +221,13 @@ The client is using device ID notifications and the server has deleted a device ID mapping held by the client. - This flag will stay in affect until the client sends GETDEVICEINFO - with a null value in the argument gdia_notify_types. + This flag will stay in effect until the client sends a GETDEVICEINFO + on the device ID with a null value in the argument gdia_notify_types. - The value of sa_sequenceid argument relative to + The value of the sa_sequenceid argument relative to the cached sequence id on the slot falls into one of three cases. @@ -244,7 +244,7 @@ If sa_sequenceid and the cached sequence id are the same, this is a retry, and the server replies with the COMPOUND reply that is stored the reply - cache. The lease is possibly renewed. + cache. The lease is possibly renewed as described below. Index: nfsv41_middle_op_set_ssv.xml =================================================================== RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_set_ssv.xml,v retrieving revision 1.28 diff -u -r1.28 nfsv41_middle_op_set_ssv.xml --- nfsv41_middle_op_set_ssv.xml 31 Mar 2008 23:18:57 -0000 1.28 +++ nfsv41_middle_op_set_ssv.xml 1 May 2008 04:07:18 -0000 @@ -30,7 +30,7 @@ the server returns NFS4ERR_INVAL in that case. - ssa_digest is computed as the output of + The field ssa_digest is computed as the output of the HMAC RFC2104 using the subkey derived from the SSV4_SUBKEY_MIC_I2T and current SSV as the key (See for a @@ -59,9 +59,9 @@ SSV version number, which is set to one (1) the first time SET_SSV executes on the server and the client receives the first SET_SSV reply. Each subsequent - SET_SSV increases the internal counter by one (1). The + SET_SSV increases the internal SSV version number by one (1). The value of this version number corresponds to the smpt_ssv_seq, - smt_ssv_seq, sspt_ssv_seq, and ssct_ssv_seq fields for the + smt_ssv_seq, sspt_ssv_seq, and ssct_ssv_seq fields of the SSV GSS mechanism tokens (see ).
@@ -83,7 +83,7 @@ Clients SHOULD NOT send an ssa_ssv that is equal to a previous - ssa_ssv, nor equal to a previous SSV (including an ssa_ssv equal to zero + ssa_ssv, nor equal to a previous or current SSV (including an ssa_ssv equal to zero since the SSV is initialized to zero when the client ID is created). Index: nfsv41_middle_op_want_delegation.xml =================================================================== RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_want_delegation.xml,v retrieving revision 1.16 diff -u -r1.16 nfsv41_middle_op_want_delegation.xml --- nfsv41_middle_op_want_delegation.xml 19 Mar 2008 05:04:46 -0000 1.16 +++ nfsv41_middle_op_want_delegation.xml 1 May 2008 04:07:18 -0000 @@ -17,30 +17,38 @@ apply, the server MAY return any of the mandated error codes. - This operation allows a client to + The server MAY support this operation. If the + server does not support this operation, it MUST + return NFS4ERR_NOTSUPP. + + + + This operation allows a client to: - get a delegation on all types - of files except directories. The server MAY support this - operation. If the server does not support this operation, it - MUST return NFS4ERR_NOTSUPP. + Get a delegation on all types + of files except directories. - register a "want" for a delegation for the specified - file object, and be notified via a callback when the delegation - is available. The server MAY support notifications of - availability via callbacks. If the server does not support - registration of wants it MUST NOT return an error to indicate - that, and instead MUST return ond_why set to WND4_CONTENTION or + Register a "want" for a delegation for the + specified file object, and be notified via a + callback when the delegation is available. The + server MAY support notifications of availability + via callbacks. If the server does not support + registration of wants it MUST NOT return + an error to indicate that, and instead MUST + return with ond_why set to WND4_CONTENTION or WND4_RESOURCE and ond_server_will_push_deleg or - ond_server_will_signal_avail set to - FALSE. When the server indicates that it will notify the client - by means of a callback, it will either provide the delegation - using a CB_PUSH_DELEG operation, or cancel its promise by - sending a CB_WANTS_CANCELLED operation. + ond_server_will_signal_avail set to FALSE. When the + server indicates that it will notify the client + by means of a callback, it will either provide + the delegation using a CB_PUSH_DELEG operation, or + cancel its promise by sending a CB_WANTS_CANCELLED + operation. + - cancel a want for a delegation. + Cancel a want for a delegation. @@ -85,8 +93,8 @@ The successful results of WANT_DELEG are of type open_delegation4 which is the same type as the "delegation" - field in the results of the OPEN operation. - (See ). + field in the results of the OPEN operation + (see ). The server constructs wdr_resok4 the same way it constructs OPEN's "delegation" with one difference: WANT_DELEGATION MUST NOT return a delegation type of Index: nfsv41_middle_pnfs.xml =================================================================== RCS file: /cvs/stdk/NFSv41/nfsv41_middle_pnfs.xml,v retrieving revision 1.78 diff -u -r1.78 nfsv41_middle_pnfs.xml --- nfsv41_middle_pnfs.xml 8 Apr 2008 17:05:19 -0000 1.78 +++ nfsv41_middle_pnfs.xml 1 May 2008 04:07:19 -0000 @@ -1186,10 +1186,10 @@ -
+
The rules for layout stateid processing differ from other stateids - in the protocol because the "seqid" value can not be zero and the + in the protocol because the "seqid" value cannot be zero and the stateid's "seqid" value changes in a CB_LAYOUTRECALL operation. The non-zero requirement combined with the inherent parallelism of layout operations means that a set of LAYOUTGET and LAYOUTRETURN @@ -1227,8 +1227,8 @@ The server MUST support a minimum VALID_SEQID_RANGE. The minimum is defined as: VALID_SEQID_RANGE = summation of 1..N of (ca_maxoperations(i) - 1) where N is the number of session fore - channels and ca_maxoperations is the value returned from - CREATE_SESSION. The reason for minus 1 is to allow for the required + channels and ca_maxoperations(i) is the value of the ca_maxoperations returned from + CREATE_SESSION of the i'th session. The reason for minus 1 is to allow for the required SEQUENCE operation. The server MAY support a VALID_SEQID_RANGE value larger than the minimum. The maximum VALID_SEQID_RANGE is (2 ^ 32 - 2) (accounts for 0 not being a valid "seqid" value).