Skip to main content

pySpark Developer with AWS EMR Experience Job in Portland, Oregon United States

Mandatory skills:
  • 5 years of experience in programming with python. Strong proficiency in python
  • Familiarity with functional programming concepts
  • 3 years of Hands on experience in developing ETL data pipelines using pyspark on AWS EMR
  • Hands on experience of XML processing using python
  • Good understanding  of Spark’s RDD API
  • Good understanding of Spark’s Dataframe and API
  • Experience in configuring EMR clusters on AWS
  • Experience and good understanding of Apache Spark Data sources API.
  • Experience of dealing with AWS S3 object storage from Spark. 
  • Experience in trouble shooting spark jobs. Knowledge of monitoring spark jobs using Spark UI
  • Performance tuning of Spark jobs.
  • Understanding fundamental design principles behind a business processes
Nice to have skills:
  • Knowledge of AWS SDK CLI
  • Experience of setting up continuous integration/deployment of spark jobs to EMR clusters
  • Knowledge of scheduling spark applications in AWS EMR cluster.
  • Understanding the differences between Hadoop Mapreduce and Apache Spark
  • Proficient understanding of code versioning tools as Git, SVN
  • Design, development and implementation of  performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR
  • Writing reusable, testable, and efficient code
  • Integration of data storage solutions in spark – especially with AWS S3 object storage.
  • Performance tuning of pySpark scripts.
  • Need to ensure overall build delivery quality is good and on time delivery is done at all times.
  • Should be able to handle meetings with customers with ease.
  • Need to have excellent communication skills to interact with customer.
  • Be a team player and willing to work in an onsite offshore model, mentor other folks in the team (onsite as well as offshore)
Send resume to


Popular posts from this blog

Administrative Assistant Job at Kazar Group Dubai, United Arab Emirates

Currently has an open position for a determined Administrative Assistant who is deeply passionate about growing their career. You will take charge of a number of different office support and administrative duties as the Administrative Assistant. Do you embrace the tasks of mail merging, pivot tables, and presentation design? Then we have just the Administrative Assistant position for you. A long-term contract / temporary to hire opportunity, this Administrative Assistant opportunity is located in the Dubai, UAE. How you will make an impact Handle telephone calls Back various projects for other employees Organize word processors, files, and faxes Greet and guide visitors Requirements At least 1 year of Administrative Assistant experience preferred Command of Clerical Duties Customer service experience Comprehensive knowledge of Administrative Assistance Proven ability to use the internet for research Knowledgeable computer skills including word processing, spreadsheets and presentation

IT Support Engineer Job in Dubai United Arab Emirates

We have Openings for IT Support Engineer in Dubai, UAE Location. Please find the details below Job Location : Dubai, UAE Position: IT Support Engineer Payroll Company: Raqmiyat. Contract Duration: 12 Months Extendable Contract. Candidates should be available inside the country UAE. Job Description:- Advanced Computer Technical skills (Hardware, Software and Windows 10 operating system and Office 365 applications) and system application. - Basic hands-on experience for Network technology and troubleshooting. - Basic hands-on experience for Radio and Telephony - Basic hands-on experience for SQL and data base troubleshooting - ITIL Certification is Must. - 6 Days a week & Should be ready to work on 24/7 Environment including night shifts. If Interested please send your updated profile and your details to