The Technical Big Data Engineer plays a critical role in the following areas: Ingesting, transforming, storing and analyzing data in Cloudera's CDH Big Data Platform environment to support the bank's strategic business goals; Working in a development team and with other teams across the SDLC phases to ensure Valley National Bank's specific standards, procedures and methodologies are adhered to in support of the bank's strategic business goals. Responsibilities include but are not limited to:
- Build scalable databases for the consumption of structured and unstructured data, work with external data providers to maximize competitor data throughout Valley National Bank.
- Import and export data between an external RDBMS and the Hadoop Cloudera cluster, including the ability to import specific subsets, change the delimiter and file format of imported data during ingest and alter the data access pattern or privileges.
- Automate data pipelines from various systems to create advanced analytics and Machine Learning.
- Ingest real-time and near real time (NRT) streaming data into HDFS, including the ability to distribute to multiple data sources and convert data from on ingest from one format to another.
- Convert data from one file format to another, Evolve an Avro or Parquet schema for performance optimization.
- Develop RESTful web services.
- Write query to filter data or to join multiple data sets.
- Agile project methodology and waterfall where required.
Apply on company website