/azure-data-lake-storage | Type: Application | PCID required: Yes
Tools
| Tool | Description |
|---|---|
azure_data_lake_storage_filesystem_create | Create Filesystem |
azure_data_lake_storage_filesystem_delete | Delete Filesystem |
azure_data_lake_storage_filesystem_list | List Filesystems |
azure_data_lake_storage_filesystem_set_properties | Set Filesystem Properties |
azure_data_lake_storage_path_create | Create File | Create Directory |
azure_data_lake_storage_path_delete | Delete File | Delete Directory |
azure_data_lake_storage_path_list | List Paths |
azure_data_lake_storage_path_read | Read File |
azure_data_lake_storage_path_update | Append Data | Flush Data | Set Properties | Set Access Control |
azure_data_lake_storage_filesystem_create
Create Filesystem Parameters:| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
filesystem | string | Yes | — | The filesystem identifier. The value must start and end with a letter or number and must contain only letters, numbers, and the dash (-) character. Consecutive dashes are not permitted. All letters must be lowercase. The value must have between 3 and 63 characters. |
resource | string | Yes | — | The value must be “filesystem” for all filesystem operations. |
timeout | integer | No | — | An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails. |
Show inputSchema
Show inputSchema
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"filesystem": {
"type": "string",
"description": "The filesystem identifier. The value must start and end with a letter or number and must contain only letters, numbers, and the dash (-) character. Consecutive dashes are not permitted. All letters must be lowercase. The value must have between 3 and 63 characters."
},
"resource": {
"type": "string",
"description": "The value must be \"filesystem\" for all filesystem operations.",
"enum": [
"filesystem"
]
},
"timeout": {
"type": "integer",
"description": "An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails."
}
},
"required": [
"PCID",
"filesystem",
"resource"
]
}
azure_data_lake_storage_filesystem_delete
Delete Filesystem Parameters:| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
filesystem | string | Yes | — | The filesystem identifier. The value must start and end with a letter or number and must contain only letters, numbers, and the dash (-) character. Consecutive dashes are not permitted. All letters must be lowercase. The value must have between 3 and 63 characters. |
resource | string | Yes | — | The value must be “filesystem” for all filesystem operations. |
timeout | integer | No | — | An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails. |
Show inputSchema
Show inputSchema
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"filesystem": {
"type": "string",
"description": "The filesystem identifier. The value must start and end with a letter or number and must contain only letters, numbers, and the dash (-) character. Consecutive dashes are not permitted. All letters must be lowercase. The value must have between 3 and 63 characters."
},
"resource": {
"type": "string",
"description": "The value must be \"filesystem\" for all filesystem operations.",
"enum": [
"filesystem"
]
},
"timeout": {
"type": "integer",
"description": "An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails."
}
},
"required": [
"PCID",
"filesystem",
"resource"
]
}
azure_data_lake_storage_filesystem_list
List Filesystems Parameters:| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
resource | string | Yes | — | The value must be “account” for all account operations. |
prefix | string | No | — | Filters results to filesystems within the specified prefix. |
continuation | string | No | — | The number of filesystems returned with each invocation is limited. If the number of filesystems to be returned exceeds this limit, a continuation token is returned in the response header x-ms-continuation. When a continuation token is returned in the response, it must be specified in a subsequent invocation of the list operation to continue listing the filesystems. |
maxResults | integer | No | — | An optional value that specifies the maximum number of items to return. If omitted or greater than 5,000, the response will include up to 5,000 items. |
timeout | integer | No | — | An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails. |
Show inputSchema
Show inputSchema
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"resource": {
"type": "string",
"description": "The value must be \"account\" for all account operations.",
"enum": [
"account"
]
},
"prefix": {
"type": "string",
"description": "Filters results to filesystems within the specified prefix."
},
"continuation": {
"type": "string",
"description": "The number of filesystems returned with each invocation is limited. If the number of filesystems to be returned exceeds this limit, a continuation token is returned in the response header x-ms-continuation. When a continuation token is returned in the response, it must be specified in a subsequent invocation of the list operation to continue listing the filesystems."
},
"maxResults": {
"type": "integer",
"description": "An optional value that specifies the maximum number of items to return. If omitted or greater than 5,000, the response will include up to 5,000 items."
},
"timeout": {
"type": "integer",
"description": "An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails."
}
},
"required": [
"PCID",
"resource"
]
}
azure_data_lake_storage_filesystem_set_properties
Set Filesystem Properties Parameters:| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
filesystem | string | Yes | — | The filesystem identifier. The value must start and end with a letter or number and must contain only letters, numbers, and the dash (-) character. Consecutive dashes are not permitted. All letters must be lowercase. The value must have between 3 and 63 characters. |
resource | string | Yes | — | The value must be “filesystem” for all filesystem operations. |
timeout | integer | No | — | An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails. |
x-ms-properties | string | No | — | User-defined properties to be stored with the filesystem, in the format of a comma-separated list of name and value pairs “n1=v1, n2=v2, …”, where each value is a base64 encoded string. Note that the string may only contain ASCII characters in the ISO-8859-1 character set. If the filesystem exists, any properties not included in the list will be removed. All properties are removed if the header is omitted. |
Show inputSchema
Show inputSchema
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"filesystem": {
"type": "string",
"description": "The filesystem identifier. The value must start and end with a letter or number and must contain only letters, numbers, and the dash (-) character. Consecutive dashes are not permitted. All letters must be lowercase. The value must have between 3 and 63 characters."
},
"resource": {
"type": "string",
"description": "The value must be \"filesystem\" for all filesystem operations.",
"enum": [
"filesystem"
]
},
"timeout": {
"type": "integer",
"description": "An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails."
},
"x-ms-properties": {
"type": "string",
"description": "User-defined properties to be stored with the filesystem, in the format of a comma-separated list of name and value pairs \"n1=v1, n2=v2, ...\", where each value is a base64 encoded string. Note that the string may only contain ASCII characters in the ISO-8859-1 character set. If the filesystem exists, any properties not included in the list will be removed. All properties are removed if the header is omitted."
}
},
"required": [
"PCID",
"filesystem",
"resource"
]
}
azure_data_lake_storage_path_create
Create File | Create Directory Parameters:| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
resource | string | No | — | Required only for Create File and Create Directory. The value must be “file” or “directory”. |
filesystem | string | Yes | — | The filesystem identifier. |
path | string | Yes | — | The file or directory path. |
timeout | integer | No | — | An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails. |
Show inputSchema
Show inputSchema
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"resource": {
"type": "string",
"description": "Required only for Create File and Create Directory. The value must be \"file\" or \"directory\".",
"enum": [
"directory",
"file"
]
},
"filesystem": {
"type": "string",
"description": "The filesystem identifier."
},
"path": {
"type": "string",
"description": "The file or directory path."
},
"timeout": {
"type": "integer",
"description": "An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails."
}
},
"required": [
"PCID",
"filesystem",
"path"
]
}
azure_data_lake_storage_path_delete
Delete File | Delete Directory Parameters:| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
recursive | boolean | No | — | Required and valid only when the resource is a directory. If “true”, all paths beneath the directory will be deleted. If “false” and the directory is non-empty, an error occurs. |
continuation | string | No | — | Optional. When deleting a directory, the number of paths that are deleted with each invocation is limited. If the number of paths to be deleted exceeds this limit, a continuation token is returned in this response header. When a continuation token is returned in the response, it must be specified in a subsequent invocation of the delete operation to continue deleting the directory. |
filesystem | string | Yes | — | The filesystem identifier. |
path | string | Yes | — | The file or directory path. |
timeout | integer | No | — | An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails. |
Show inputSchema
Show inputSchema
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"recursive": {
"type": "boolean",
"description": "Required and valid only when the resource is a directory. If \"true\", all paths beneath the directory will be deleted. If \"false\" and the directory is non-empty, an error occurs."
},
"continuation": {
"type": "string",
"description": "Optional. When deleting a directory, the number of paths that are deleted with each invocation is limited. If the number of paths to be deleted exceeds this limit, a continuation token is returned in this response header. When a continuation token is returned in the response, it must be specified in a subsequent invocation of the delete operation to continue deleting the directory."
},
"filesystem": {
"type": "string",
"description": "The filesystem identifier."
},
"path": {
"type": "string",
"description": "The file or directory path."
},
"timeout": {
"type": "integer",
"description": "An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails."
}
},
"required": [
"PCID",
"filesystem",
"path"
]
}
azure_data_lake_storage_path_list
List Paths Parameters:| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
directory | string | No | — | Filters results to paths within the specified directory. An error occurs if the directory does not exist. |
recursive | boolean | Yes | — | If “true”, all paths are listed; otherwise, only paths at the root of the filesystem are listed. If “directory” is specified, the list will only include paths that share the same root. |
continuation | string | No | — | The number of paths returned with each invocation is limited. If the number of paths to be returned exceeds this limit, a continuation token is returned in the response header x-ms-continuation. When a continuation token is returned in the response, it must be specified in a subsequent invocation of the list operation to continue listing the paths. |
maxResults | integer | No | — | An optional value that specifies the maximum number of items to return. If omitted or greater than 5,000, the response will include up to 5,000 items. |
upn | boolean | No | — | Optional. Valid only when Hierarchical Namespace is enabled for the account. If “true”, the user identity values returned in the owner and group fields of each list entry will be transformed from Azure Active Directory Object IDs to User Principal Names. If “false”, the values will be returned as Azure Active Directory Object IDs. The default value is false. Note that group and application Object IDs are not translated because they do not have unique friendly names. |
filesystem | string | Yes | — | The filesystem identifier. The value must start and end with a letter or number and must contain only letters, numbers, and the dash (-) character. Consecutive dashes are not permitted. All letters must be lowercase. The value must have between 3 and 63 characters. |
resource | string | Yes | — | The value must be “filesystem” for all filesystem operations. |
timeout | integer | No | — | An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails. |
Show inputSchema
Show inputSchema
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"directory": {
"type": "string",
"description": "Filters results to paths within the specified directory. An error occurs if the directory does not exist."
},
"recursive": {
"type": "boolean",
"description": "If \"true\", all paths are listed; otherwise, only paths at the root of the filesystem are listed. If \"directory\" is specified, the list will only include paths that share the same root."
},
"continuation": {
"type": "string",
"description": "The number of paths returned with each invocation is limited. If the number of paths to be returned exceeds this limit, a continuation token is returned in the response header x-ms-continuation. When a continuation token is returned in the response, it must be specified in a subsequent invocation of the list operation to continue listing the paths."
},
"maxResults": {
"type": "integer",
"description": "An optional value that specifies the maximum number of items to return. If omitted or greater than 5,000, the response will include up to 5,000 items."
},
"upn": {
"type": "boolean",
"description": "Optional. Valid only when Hierarchical Namespace is enabled for the account. If \"true\", the user identity values returned in the owner and group fields of each list entry will be transformed from Azure Active Directory Object IDs to User Principal Names. If \"false\", the values will be returned as Azure Active Directory Object IDs. The default value is false. Note that group and application Object IDs are not translated because they do not have unique friendly names."
},
"filesystem": {
"type": "string",
"description": "The filesystem identifier. The value must start and end with a letter or number and must contain only letters, numbers, and the dash (-) character. Consecutive dashes are not permitted. All letters must be lowercase. The value must have between 3 and 63 characters."
},
"resource": {
"type": "string",
"description": "The value must be \"filesystem\" for all filesystem operations.",
"enum": [
"filesystem"
]
},
"timeout": {
"type": "integer",
"description": "An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails."
}
},
"required": [
"PCID",
"recursive",
"filesystem",
"resource"
]
}
azure_data_lake_storage_path_read
Read File Parameters:| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
filesystem | string | Yes | — | The filesystem identifier. |
path | string | Yes | — | The file or directory path. |
timeout | integer | No | — | An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails. |
Show inputSchema
Show inputSchema
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"filesystem": {
"type": "string",
"description": "The filesystem identifier."
},
"path": {
"type": "string",
"description": "The file or directory path."
},
"timeout": {
"type": "integer",
"description": "An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails."
}
},
"required": [
"PCID",
"filesystem",
"path"
]
}
azure_data_lake_storage_path_update
Append Data | Flush Data | Set Properties | Set Access Control Parameters:| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
action | string | Yes | — | The action must be “append” to upload data to be appended to a file, “flush” to flush previously uploaded data to a file, “setProperties” to set the properties of a file or directory, or “setAccessControl” to set the owner, group, permissions, or access control list for a file or directory. Note that Hierarchical Namespace must be enabled for the account in order to use access control. Also note that the Access Control List (ACL) includes permissions for the owner, owning group, and others, so the x-ms-permissions and x-ms-acl request headers are mutually exclusive. |
position | integer | No | — | This parameter allows the caller to upload data in parallel and control the order in which it is appended to the file. It is required when uploading data to be appended to the file and when flushing previously uploaded data to the file. The value must be the position where the data is to be appended. Uploaded data is not immediately flushed, or written, to the file. To flush, the previously uploaded data must be contiguous, the position parameter must be specified and equal to the length of the file after all data has been written, and there must not be a request entity body included with the request. |
retainUncommittedData | boolean | No | — | Valid only for flush operations. If “true”, uncommitted data is retained after the flush operation completes; otherwise, the uncommitted data is deleted after the flush operation. The default is false. Data at offsets less than the specified position are written to the file when flush succeeds, but this optional parameter allows data after the flush position to be retained for a future flush operation. |
close | boolean | No | — | Azure Storage Events allow applications to receive notifications when files change. When Azure Storage Events are enabled, a file changed event is raised. This event has a property indicating whether this is the final change to distinguish the difference between an intermediate flush to a file stream and the final close of a file stream. The close query parameter is valid only when the action is “flush” and change notifications are enabled. If the value of close is “true” and the flush operation completes successfully, the service raises a file change notification with a property indicating that this is the final update (the file stream has been closed). If “false” a change notification is raised indicating the file has changed. The default is false. This query parameter is set to true by the Hadoop ABFS driver to indicate that the file stream has been closed.” |
filesystem | string | Yes | — | The filesystem identifier. |
path | string | Yes | — | The file or directory path. |
timeout | integer | No | — | An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails. |
x-ms-properties | string | No | — | Valid only for setProperties action. User-defined properties to be stored with the file or directory, in the format of a comma-separated list of name and value pairs “n1=v1, n2=v2, …”, where each value is a base64 encoded string. Note that the string may only contain ASCII characters in the ISO-8859-1 character set. |
x-ms-permissions | string | No | — | Valid only for setAccessControl and setProperties actions. Optional and only valid if Hierarchical Namespace is enabled for the account. Sets POSIX access permissions for the file owner, the file owning group, and others. Each class may be granted read, write, or execute permission. The sticky bit is also supported. Both symbolic (rwxrw-rw-) and 4-digit octal notation (e.g. 0766) are supported. Invalid in conjunction with x-ms-acl. |
x-ms-acl | string | No | — | Valid only for setAccessControl action. Sets POSIX access control rights on files and directories. The value is a comma-separated list of access control entries. Each entry has scope (default), type (user|group|other|mask), optional UPN or OID, and permissions (rwx or octal). Example: “user::rwx,group::r—,other::---”. Invalid in conjunction with x-ms-permissions. |
x-ms-owner | string | No | — | Valid only for setAccessControl action. Sets the owner of the file or directory. |
x-ms-group | string | No | — | Valid only for setAccessControl action. Sets the owning group of the file or directory. |
body | string | No | — | Valid only for append operations. The data to be uploaded and appended to the file. |
Show inputSchema
Show inputSchema
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"action": {
"type": "string",
"description": "The action must be \"append\" to upload data to be appended to a file, \"flush\" to flush previously uploaded data to a file, \"setProperties\" to set the properties of a file or directory, or \"setAccessControl\" to set the owner, group, permissions, or access control list for a file or directory. Note that Hierarchical Namespace must be enabled for the account in order to use access control. Also note that the Access Control List (ACL) includes permissions for the owner, owning group, and others, so the x-ms-permissions and x-ms-acl request headers are mutually exclusive.",
"enum": [
"append",
"flush",
"setProperties",
"setAccessControl"
]
},
"position": {
"type": "integer",
"description": "This parameter allows the caller to upload data in parallel and control the order in which it is appended to the file. It is required when uploading data to be appended to the file and when flushing previously uploaded data to the file. The value must be the position where the data is to be appended. Uploaded data is not immediately flushed, or written, to the file. To flush, the previously uploaded data must be contiguous, the position parameter must be specified and equal to the length of the file after all data has been written, and there must not be a request entity body included with the request."
},
"retainUncommittedData": {
"type": "boolean",
"description": "Valid only for flush operations. If \"true\", uncommitted data is retained after the flush operation completes; otherwise, the uncommitted data is deleted after the flush operation. The default is false. Data at offsets less than the specified position are written to the file when flush succeeds, but this optional parameter allows data after the flush position to be retained for a future flush operation."
},
"close": {
"type": "boolean",
"description": "Azure Storage Events allow applications to receive notifications when files change. When Azure Storage Events are enabled, a file changed event is raised. This event has a property indicating whether this is the final change to distinguish the difference between an intermediate flush to a file stream and the final close of a file stream. The close query parameter is valid only when the action is \"flush\" and change notifications are enabled. If the value of close is \"true\" and the flush operation completes successfully, the service raises a file change notification with a property indicating that this is the final update (the file stream has been closed). If \"false\" a change notification is raised indicating the file has changed. The default is false. This query parameter is set to true by the Hadoop ABFS driver to indicate that the file stream has been closed.\""
},
"filesystem": {
"type": "string",
"description": "The filesystem identifier."
},
"path": {
"type": "string",
"description": "The file or directory path."
},
"timeout": {
"type": "integer",
"description": "An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails."
},
"x-ms-properties": {
"type": "string",
"description": "Valid only for setProperties action. User-defined properties to be stored with the file or directory, in the format of a comma-separated list of name and value pairs \"n1=v1, n2=v2, ...\", where each value is a base64 encoded string. Note that the string may only contain ASCII characters in the ISO-8859-1 character set."
},
"x-ms-permissions": {
"type": "string",
"description": "Valid only for setAccessControl and setProperties actions. Optional and only valid if Hierarchical Namespace is enabled for the account. Sets POSIX access permissions for the file owner, the file owning group, and others. Each class may be granted read, write, or execute permission. The sticky bit is also supported. Both symbolic (rwxrw-rw-) and 4-digit octal notation (e.g. 0766) are supported. Invalid in conjunction with x-ms-acl."
},
"x-ms-acl": {
"type": "string",
"description": "Valid only for setAccessControl action. Sets POSIX access control rights on files and directories. The value is a comma-separated list of access control entries. Each entry has scope (default), type (user|group|other|mask), optional UPN or OID, and permissions (rwx or octal). Example: \"user::rwx,group::r--,other::---\". Invalid in conjunction with x-ms-permissions."
},
"x-ms-owner": {
"type": "string",
"description": "Valid only for setAccessControl action. Sets the owner of the file or directory."
},
"x-ms-group": {
"type": "string",
"description": "Valid only for setAccessControl action. Sets the owning group of the file or directory."
},
"body": {
"type": "string",
"description": "Valid only for append operations. The data to be uploaded and appended to the file."
}
},
"required": [
"PCID",
"action",
"filesystem",
"path"
]
}