Skip to main content

pySpark Developer with AWS EMR Experience Job in Portland, Oregon United States

Mandatory skills:
  • 5 years of experience in programming with python. Strong proficiency in python
  • Familiarity with functional programming concepts
  • 3 years of Hands on experience in developing ETL data pipelines using pyspark on AWS EMR
  • Hands on experience of XML processing using python
  • Good understanding  of Spark’s RDD API
  • Good understanding of Spark’s Dataframe and API
  • Experience in configuring EMR clusters on AWS
  • Experience and good understanding of Apache Spark Data sources API.
  • Experience of dealing with AWS S3 object storage from Spark. 
  • Experience in trouble shooting spark jobs. Knowledge of monitoring spark jobs using Spark UI
  • Performance tuning of Spark jobs.
  • Understanding fundamental design principles behind a business processes
Nice to have skills:
  • Knowledge of AWS SDK CLI
  • Experience of setting up continuous integration/deployment of spark jobs to EMR clusters
  • Knowledge of scheduling spark applications in AWS EMR cluster.
  • Understanding the differences between Hadoop Mapreduce and Apache Spark
  • Proficient understanding of code versioning tools as Git, SVN
  • Design, development and implementation of  performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR
  • Writing reusable, testable, and efficient code
  • Integration of data storage solutions in spark – especially with AWS S3 object storage.
  • Performance tuning of pySpark scripts.
  • Need to ensure overall build delivery quality is good and on time delivery is done at all times.
  • Should be able to handle meetings with customers with ease.
  • Need to have excellent communication skills to interact with customer.
  • Be a team player and willing to work in an onsite offshore model, mentor other folks in the team (onsite as well as offshore)
Send resume to


Popular posts from this blog

Customers Service Executive Job in Abu Dhabi United Arab Emirates

Customers Service Executive (Abu Dhabi) - Parking and Removing / Relocation experience mandatory. Checking & verifying with sales dept for freight rates for processing import shipment • Coordinating with the customs clearance assistant and airlines for the collection of Delivery Orders from DNATA. • Handling sea freight import FCL & LCL containers • Liaising with shipping lines/agents, DPA, Customs authority for LCL/FCL or co-load type of shipments • Monitor various shipping movement and keep a track of all shipments & vessel arrival. • To make import bookings for warehouse & set priorities with regular follow-up for urgent deliveries • Ensure timely availability of shipments to customers within the time frame. • Co-ordination with overseas partner & airline agents. • Follow-up for NOC (No Objection Letter) from the liner agents and collect MR number from DPA • Sending arrival notification to customers • Checking & verifying with sales dept for freight rates f

F and b Manager Job at Prime Immigration LLP Dubai, United Arab Emirates

Job Description Responsible for ensuring that all of the food and drinks are of the highest quality. A few of the main duties are designing unique menus, handling customer complaints, complying with food safety regulations. Required Candidate Profile Responsible for overall operation for the restaurant, FB managers purchase food stock, make sure everyone is trained, proper legal alcoholic beverage service kitchen safety health etc. Send resume to adelina@

Key Account Manager (Dubai) Job at Air Arabia Dubai United Arab Emirates

Job Description Air Arabia is not only committed to providing affordable air travel but is also dedicated to uplifting the lives of those who are less fortunate. Taking responsibility and lead on social needs of local and international communities has been part of our success. Job Purpose To achieve the commercial objectives and targets in the assigned area/territory from different channels: building effective business relationships, enhancing brand visibility and promoting Company’s products and services. Carries marketing campaigns and promotions, and increases the market share in line with set measures and Company’s adopted policies and procedures. Key Result Responsibilities Achieves and exceeds the set sales targets in the assigned area/territory; constantly reviews financial data to evaluate performance; identifies shortages, determines root causes, recommends changes to respective Manager and implements corrective measures accordingly. Identifies new opportunities to manage/maxi