BIG EFFECT®Remote
Full Stack Data Engineer
United States of America
Full Time
3 hours ago
$75 - $85 USD
Visa Sponsor
Key skills
PythonSQLScalaJavaSupabaseSnowflakeAWSGCPAzureETL pipelinesELT pipelinesData transformationData modelingREST APIsGraphQL APIsFastAPIDockerKubernetesCI/CDReactStreamlitMetabaseD3.jsVega-AltairPlotlyNeo4jAmazon NeptuneArangoDBCypherGremlinMusic streaming dataSocial media data processingGoogle AdsMeta Ads ManagerDSPsDifferential privacyClean roomsCookieless attributionML pipelinesFeature storesMLOpsMLETLELTData EngineeringData LakeAnalyticsGraphQLRESTfulCaching
About this role
BIG EFFECT® is seeking a Full Stack Data Engineer to design, build, and maintain scalable data pipelines and architecture. The role involves developing APIs, managing data quality, and overseeing infrastructure and DevOps practices.
Responsibilities:
- Pipeline design & orchestration
- Design, build, and maintain scalable batch and streaming data pipelines. Orchestrate workflows, ensuring fault-tolerance, observability, and automated recovery
- Data warehouse & lake architecture
- Model and manage data in cloud data warehouses and data lake environments. Own schema design, partitioning strategies, and incremental loading patterns to optimize cost and query performance
- API & backend data services
- Develop and maintain RESTful or GraphQL APIs that serve processed data to internal and external consumers. Design low-latency serving layers and caching strategies to support high-traffic product features
- Frontend data tooling & dashboards
- Build internal and customer facing data tools, dashboards, and self-serve analytics interfaces. Partner with analysts, data scientists and product engineers to translate data needs into intuitive user experiences
- Data quality & governance
- Implement data quality frameworks, testing strategies, and monitoring alerting across pipelines. Contribute to data cataloging, lineage tracking, and access control policies in line with compliance requirements
- Infrastructure & DevOps
- Manage data infrastructure. Own CI/CD for data pipelines, containerize workloads with Docker and Kubernetes, and help define infrastructure standards for the data platform team
Requirements:
- 5+ years of software or data engineering experience with production systems at scale
- Strong proficiency in Python and SQL; comfort with Scala or Java a plus
- Hands-on experience with a cloud data platforms (Supabase, Snowflake, AWS/GCP/AZure)
- Expertise in building and operating ETL/ELT pipelines in batch and/or streaming contexts
- Expertise in data transformation and modeling
- Expertise in developing APIs or microservices (REST, GraphQL, FastAPI, or similar)
- Comfortable with containerization (Docker, Kubernetes) and CI/CD practices
- Ability to build data dashboards or internal tools using React, Streamlit, Metabase, or similar
- Familiarity with data visualization libraries (D3.js, Vega-Altair, Plotly)
- Hands-on experience with graph databases (e.g. Neo4j, Amazon Neptune, or ArangoDB) and graph query languages such as Cypher or Gremlin; ability to model complex entity relationships as graphs and run traversal-based analytics at scale
- Prior experience in music streaming data, artist and track metadata, audio feature datasets, listener behavior data, or recommendation signal pipelines at consumer scale
- Experience ingesting and processing social media data — including engagement signals, user-generated content, follower graphs, and platform API integrations (e.g., Meta, TikTok, X/Twitter, YouTube)
- Familiarity with marketing and advertising data domains — including campaign attribution, conversion funnels, audience segmentation, impression and click-through data, and integration with ad platforms such as Google Ads, Meta Ads Manager, or DSPs
- Experience with privacy-safe data practices relevant to advertising (e.g., differential privacy, clean rooms, cookieless attribution)
- Experience with ML pipelines, feature stores, or MLOps tooling