HomeTamr Core GuidesTamr Core API Reference
Tamr Core GuidesTamr Core API ReferenceTamr Core TutorialsEnrichment API ReferenceSupport Help CenterLog In

External Storage Providers

Configure external storage providers.

To create a dataset for upload to, or export from, Tamr that is not stored in the Tamr primary storage space, you configure an external storage provider for that dataset.

Configuring External Storage Providers

Use TAMR_STORAGE_PROVIDERS:

Configuration Variable

Example Value

TAMR_STORAGE_PROVIDERS

[{
  "name": "storage-provider-one",
  "description": "my local filesystem storage",
  <storage-provider-type>
},{
  "name": "storage-provider-two",
  "description": "my hdfs storage",
  <storage-provider-type>
},
...
]
  1. Set a value for the TAMR_STORAGE_PROVIDERS configuration variable using the admin tool. See Creating or Updating a Configuration Variable.
    <storage-provider-type> may be HDFS. For details, see Configuring an HDFS External Storage Provider.
  2. Restart Tamr and its dependencies. See Restarting.

Configuring an HDFS External Storage Provider

Json Key

Description / Example Value

hdfs.fsUri

See TAMR_FS_URI.

hdfs.configFiles

See TAMR_FS_CONFIG_URIS.

hdfs.extraFiles

See TAMR_FS_EXTRA_URIS.

hdfs.configDirectory

See TAMR_FS_CONFIG_DIR.

hdfs.overrideFsConfig

See TAMR_FS_EXTRA_CONFIG.

hdfs.kerberos.principal

(Optional)
See TAMR_KERBEROS_PRINCIPAL.

hdfs.kerberos.keytab

(Optional)
See TAMR_KERBEROS_KEYTAB.

hdfs.kerberos.krb5

(Optional)
See TAMR_FS_KERBEROS_KRB5.

To configure an HDFS external storage provider:

  1. In addition to the JSON keys name and description, set values for each of the previously-listed JSON keys to create the HDFS storage provider JSON object. For example:
[{
  "name": "providerName",
  "description": "my HDFS provider",
  "hdfs": {
    "fsUri": "hdfs://nameservice",
    "configFiles": "file://path/to/core-site.xml;file://path/to/hdfs-site.xml",
    "extraFiles": "file://path/to/non-xml-files.txt",
    "configDirectory": "/data/fs/hdfs/conf",
    "overrideFsConfig": {},
    "kerberos": {
      "principal": "primary/[email protected]",
      "keytab": "/path/to/keytab"
    }
  }
}]

Did this page help you?