Software Engineer-SE - AWS Big Data

NEC Software Solutions (India) 

On 1st July 2021, Rave Technologies became NEC Software Solutions India. This change brought us under the global NEC Corporation brand. We are proud to be part of an organisation with 122 years of experience in evolution with technology and innovation.

We have more than 30 years of experience in providing end to end IT services across the globe and have earned a reputation for delighting our customers by consistently surpassing expectations and helping them deliver robust, market-ready software products that meet the highest standards of engineering and user experience. Supported by more than 1300 exceptionally talented manpower, we are a hub for offshore support and technology services.

We work with diverse industry verticals which include publishing, media, financial services, retail, healthcare and technology companies around the world. Our customers range from two-person startups to $bn listed companies.

For more information, visit at www.necsws.com/india.

About NEC Corporation 

NEC Corporation is a Japanese multinational information technology and electronics company, headquartered in Tokyo, Japan. It is recognised as a ‘Top 50 Innovative Company’ globally and the NEC Group globally provides “Solutions for Society” that promote the safety, security, fairness and equality of society. Their main goal is to help create a safer society with their innovations in technologies.

NEC Corporation has established itself as a leader in the integration of IT and network technologies while promoting the brand statement of “Orchestrating a brighter world.” NEC enables businesses and communities to adapt to rapid changes taking place in both society and the market as it provides for the social values of safety, security, fairness and efficiency to promote a more sustainable world where everyone has the chance to reach their full potential. 

For more information, visit NEC at https://www.nec.com.  

2-5 years experience as a AWS Big data engineer

Expert in SQL (Postgres / Redshift)

Expertise in AWS Glue, Athena and Spark

Expertise in working with AWS Kinesis

Working with EMR

Expert in SQL (Postgres / Redshift)

• Query factoring (With clause, Common Table Expression)
• Window functions: rank, row_number, lead, lag, first_value, last_value
• Sub-Queries
• Joins: Self, Inner, Outer (Left/Right), Full-Outer
• group by, having clause, orde by etc
• External Tables
• OLAP Cubes and Rollupso   
• Bulk load using COPY and Bulk download using UNLOAD
• Stored Procedures
• Transaction Management
• Incremental loading of data from s3 files using upsert/merge
• Using explain plan to improve query performance
• Using incremental refresh Materialized Views
• Effectively Using DISTKEY and SORTKEY to achieve high performance distributed queries
• Excel like Pivot and Unpivot columns in SQL queries

Expertise in AWS Glue, Athena and Spark

• Writing Spark batch and Streaming jobs (Python or Scala)
• Performance tuning of spark jobs
• Stream aggregation using kinesis /kafka and writing to redshift
• Reading and writing partitioned S3 data
• Using glue to join disparate data sources like RDS, Redshift, S3 etc.
• Using Athena to query S3 data.
• Process distributed data sources using Spark
• Writing glue pipelines

 Redshift

• Managing clusters
• Creating Cluster
• Scaling: Elastic / Classic
• Taking snapshots
• Restoring clusters from snapshots
• Monitoring CPU/RAM etc

Expertise in working with AWS Kinesis

Working with EMR

Creating transient as well as persistent clusters

Reading and writing S3 files from EMR Spark jobs

Good Communication Skills required.