πŸ“œ
Our Manifesto
🧰
Backup & Disaster Recovery
πŸ‘¨β€πŸ‘©β€πŸ‘§β€πŸ‘¦ Customer Success & Support
πŸ‘¨β€πŸ‘©β€πŸ‘§β€πŸ‘¦ Community
Hive
Steps to integrate your Hive with Atlan
Atlan natively supports the Apache Hive metastore which allows you to seamlessly integrate your metadata with your Atlan Workspace.

πŸ’­ TL;DR

You can set up a Hive integration with your Atlan workspace in 4 easy steps:
  1. 1.
    Select the Source aka Hive πŸ˜‰
  2. 2.
    Provide your credentials ✍️
  3. 3.
    Set up your configuration πŸ—„οΈ
  4. 4.
    Schedule automatic updates πŸ•‘

πŸ“œ Prerequisites for Hive Integration

Before you get started with integrating your Apache Hive with Atlan, you'll need some prerequisite information which will help establish a connection between Atlan and your Hive Account:
  • Hostname - Hostname is the IP address of the Hive server to which you are connecting.
  • Hive Server Port - is the number of the TCP port that the Hive server machine uses to listen for client connections. The default Hive server port is 10000
  • Hive Metastore Port - Server port used for accessing metadata about hive tables and partitions. The default Hive metastore port is 9083
  • AWS Access key & Secret Key - Access keys consist of an access key ID and a secret access key, which are used to sign programmatic requests that you make to AWS. Visit AWS documentation around Access keys to know more about access keys and how to create them.
    • S3 Permissions required for enabling data profile.
      1
      {
      2
      "Version": "2012-10-17",
      3
      "Statement": [
      4
      {
      5
      "Sid": "VisualEditor0",
      6
      "Effect": "Allow",
      7
      "Action": [
      8
      "s3:GetObjectAcl",
      9
      "s3:GetObject",
      10
      "s3:ListBucketMultipartUploads",
      11
      "s3:ListBucketVersions",
      12
      "s3:ListBucket",
      13
      "s3:GetBucketAcl",
      14
      "s3:GetFileStatus",
      15
      "s3:ListMultipartUploadParts"
      16
      ],
      17
      "Resource": [
      18
      "arn:aws:s3:::<bucket-name>",
      19
      "arn:aws:s3:::<bucket-name>/*"
      20
      ]
      21
      },
      22
      {
      23
      "Sid": "VisualEditor1",
      24
      "Effect": "Allow",
      25
      "Action": [
      26
      "s3:ListAllMyBuckets",
      27
      "s3:ListAccessPoints",
      28
      "s3:ListJobs",
      29
      "s3:CreateJob",
      30
      "s3:HeadBucket"
      31
      ],
      32
      "Resource": "*"
      33
      }
      34
      ]
      35
      }
      Copied!
  • Default Schema - The schema used to store the Hive table. If not sure, use default.
🌟 Pro Tip: If you don't have this information handy, reach out to your cloud or data lake administrator to get these details before you get started!

πŸš€ The step-by-step guide to integrate Hive with Atlan

Once you have the prerequisite information listed in the section above, please follow the steps below to establish a connection and integrate Atlan with your Hive metastore.

STEP 1: Selecting the Source

  1. 1.
    Log into your Atlan Workspace
  2. 2.
    On the Home Screen, click on the "New Integration" button in the top right corner. You will see a Dialogue box with the list of sources available on your workspace
  3. 3.
    Select "Hive" from the list of options and click on "Next"

STEP 2: Providing Credentials

  1. 1.
    You will see an option to either select a pre-configured credential from the drop-down or to create a credential**.** To set up a new connection, click on the "Create Credential" button.
  2. 2.
    You will be required to fill in your Hive credentials. Below is an example of the credentials required: Hostname - 1.1.1.1 Hive server Port - 10000 Hive metastore port - 9083 Schema - default AWS Access key - AKIA5XXXXXXXXXXWIJUS AWS Secret key - R1xXXXXXXXXX5PEdHOUXXXXXXXX7Ooz47
  3. 3.
    Once you have filled in the details, click on "Next".

STEP 3: Setting up Configuration

  1. 1.
    You will now be asked to fill in the details of your database and table. You can also choose the entire schema and table by selecting the checkbox. Below is an example - Add Schema - Sales Master Add Table - Daily Sales
  2. 2.
    Chose whether to run the crawler once or schedule it for a Daily, a Weekly, or a Monthly run. You would be asked to specify the time zone to trigger the run.
  3. 3.
    Click on "Create". Your connection is now created.
Congratulations, you have now integrated Atlan with your Hive metastore! πŸŽ‰

🏁 Monitoring your Hive metastore integration

Once the integration setup is completed, you will be redirected to the Monitor tab for your Hive asset where you can monitor the progress.
Last modified 3mo ago