External Storage Providers

To create a dataset for upload or export to and from Tamr, that is not stored in the Tamr primary storage space, configure an external storage provider for that dataset.

Configuring External Storage Providers

Use TAMR_STORAGE_PROVIDERS:

Configuration Variable	Example Value
TAMR_STORAGE_PROVIDERS	`json [{ "name": "storage-provider-one", "description": "my local filesystem storage", <storage-provider-type> },{ "name": "storage-provider-two", "description": "my hdfs storage", <storage-provider-type> }, ... ]`

Set a value for the TAMR_STORAGE_PROVIDERS configuration variable using the admin tool. See Creating or Updating a Configuration Variable. <storage-provider-type-key> may be HDFS. See Configuring an HDFS External Storage Provider.
Restart Tamr and its dependencies. See Restarting.

Configuring an HDFS External Storage Provider

Json Key	Description / Example Value
`hdfs.fsUri`	See TAMR_FS_URI.
`hdfs.configFiles`	See TAMR_FS_CONFIG_URIS.
`hdfs.extraFiles`	See TAMR_FS_EXTRA_URIS.
`hdfs.configDirectory`	See TAMR_FS_CONFIG_DIR.
`hdfs.overrideFsConfig`	See TAMR_FS_EXTRA_CONFIG.
`hdfs.kerberos.principal`	(Optional) See TAMR_KERBEROS_PRINCIPAL.
`hdfs.kerberos.keytab`	(Optional) See TAMR_KERBEROS_KEYTAB.
`hdfs.kerberos.krb5`	(Optional) See TAMR_FS_KERBEROS_KRB5.

To configure an HDFS external storage provider:

In addition to the JSON keys name and description, set values for each of the previously-listed JSON keys to create the HDFS storage provider JSON object. For example:

[{
  "name": "providerName",
  "description": "my HDFS provider",
  "hdfs": {
    "fsUri": "hdfs://nameservice",
    "configFiles": "file://path/to/core-site.xml;file://path/to/hdfs-site.xml",
    "extraFiles": "file://path/to/non-xml-files.txt",
    "configDirectory": "/data/fs/hdfs/conf",
    "overrideFsConfig": {},
    "kerberos": {
      "principal": "primary/instance@REALM",
      "keytab": "/path/to/keytab"
    }
  }
}]