GEN AI - (CREQ257802)

Description

Modeling — Kimball dimensional, Data Vault 2.0, normalized 3NF, medallion (bronze/silver/gold), semantic / metrics layer; SCD types, slowly-changing dimensions, late-arriving facts, conformed dimensions.
SQL at expert level across at least one of Snowflake, BigQuery, Databricks SQL, Synapse / Fabric, Redshift, Postgres — window functions, CTEs, query optimization, partitioning / clustering, materialized views.
Distributed processing — Apache Spark / PySpark (production-scale), Databricks (Delta Lake, Unity Catalog, DLT, Workflows), Apache Flink / Beam / Dataflow awareness; performance tuning (skew, spill, shuffle, broadcast).
Streaming & CDC — Apache Kafka (Confluent / MSK), Kinesis / Pub/Sub / Event Hubs, Structured Streaming, Flink, Debezium, Fivetran / Airbyte / Qlik Replicate.
Transformation — dbt (Core / Cloud) at production scale; testing, exposures, sources, snapshots, packages, semantic models.
Orchestration — Apache Airflow (production), Dagster, Prefect, Azure Data Factory, Databricks Workflows, AWS Step Functions / Glue Workflows.
Lakehouse / formats — Delta Lake, Apache Iceberg, Apache Hudi; Parquet / ORC / Avro; partitioning, Z-ordering, OPTIMIZE / VACUUM, compaction.
Catalog & governance — Unity Catalog, Microsoft Purview, AWS Glue Catalog / Lake Formation, Collibra / Alation / OpenMetadata / DataHub; lineage (OpenLineage / Marquez), classification, data contracts.
Data quality — Great Expectations, Soda, dbt tests, Monte Carlo / Bigeye / Anomalo awareness; reconciliation, drift, freshness, completeness, uniqueness, referential integrity.
Master Data Management — Reltio / Informatica MDM / Ataccama / Stibo (engagement-dependent).
Vector & GenAI data foundations — vector stores (pgvector / Pinecone / Weaviate / Milvus / Azure AI Search / OpenSearch / Vertex Vector Search); embedding pipelines, RAG indexing, chunking / re-ranking awareness; feature stores (Feast / Tecton / Databricks Feature Store).
Storage & file formats — S3 / ADLS Gen2 / GCS, Parquet / ORC / Avro / Delta / Iceberg / Hudi.

Primary Location

: IN-AP-Hyderabad

Schedule

: Full Time

Employee Status

: Individual Contributor

Job Type

: Experienced

Travel

: No

Job Posting

: 10/07/2026, 9:44:38 AM

Gen AI Engineer

About this role