Scienaptic is a new age AI powered credit underwriting company that offers end-to-end credit decisioning platform. Scienaptic’s Ether platform is fundamentally disrupting the way consumer credit is administered.
Scienaptic is looking for a Data Engineer to join their growing team of analytics experts.
Leverage the batch computation frameworks and our workflow management platform (Airflow) to assist in building out different data pipelines
Lower the latency and bridge the gap between our production systems and our data warehouse by rethinking and optimizing our core data pipeline jobs
Work with client to create and optimize critical batch processing jobs in Spark
Develop production grade code using Scala/Spark and Python/Spark code on Azure data bricks
Skills and Experience
Strong engineering background and interested in data
Good understanding of data analysis using SQL queries
Strong hold on Python or Scala as a programming language on Azure Databricks.
Experience of developing and maintaining distributed systems built with Azure Databricks or native Apache Spark
Experience of building libraries and tooling that provide abstractions to users for accessing data
Experience in writing and debugging ETL jobs using a distributed data framework (Spark/Hadoop MapReduce etc.) on Azure Databricks
Experience optimizing the end-to-end performance of distributed systems
Ability to recommend and implement ways to improve data reliability, efficiency, and quality