Big Data Developer
United States | Canada | Remote | Work from Home
Are you a United States or Canada based Big Data Engineer or Developer who can plan, design and implement new solutions using modern Big Data technologies? Do you want to implement and help develop a cutting edge SaaS product like Pythian’s analytics as a service KickAaaS? A fully-managed, end-to-end service that brings multi-source, multi-format data together in the cloud with advanced analytics and visualizations. Can you process specifications designed by a Data Architect or Business analyst to create data pipelines which will migrate data from customers on-prem systems and load it into a cloud hosted system like KickAaaS? Do you have an interest in helping to mature a product by working with the R & D team, providing feedback and new ideas? If you have a passion for big data and solving complex problems then this could be the job for you!
What will you be doing?
- Working with the Data Architects to implement data pipelines
- Working with our Big Data Principal Architects in the development both proof of concepts and complete implementations.
- Working on complex and varied Big Data projects including tasks such as collecting, parsing, managing, analyzing, and visualizing very large datasets.
- Translating complex functional and technical requirements into detailed designs.
- Writing high-performance, reliable and maintainable code.
- Performing data processing requirements analysis.
- Performance tuning for batch and real-time data processing.
- Securing components of clients’ Big Data platforms.
- Diagnostics and troubleshooting of operational issues.
- Health-checks and configuration reviews.
- Data pipelines development – ingestion, transformation, cleansing.
- Data flow integration with external systems.
- Integration with data access tools and products.
- Assisting application developers and advising on efficient data access and manipulations.
- Defining and implementing efficient operational processes
- Dogfooding our product is important, so a short (~3 month) on call rotation with second level support is required for all team members. This works out to needing to be on call less than once a year and ensures quality of implementations.
What do we need from you?
While we realise you might not have everything on the list to be the successful candidate for the Big Data Developer job you will likely have at least 5 years experience in similar roles. The position requires specialized knowledge and experience in performing the following:
- Experience building data pipelines in any public cloud (GCP Dataflow, Glue, Azure DataFactory) or any equivalent
- Experience writing ETL (Any popular tools)
- Experience in data modeling, data design and persistence (e.g. warehousing, data marts, data lakes).
- Strong Knowledge of Big Data architectures and distributed data processing frameworks: Hadoop, Spark, Kafka, Hive
- Experience and working knowledge of various development platforms, frameworks and languages such as Java, Python, and SQL
- General knowledge of modern data-center and cloud infrastructure including server hardware, networking and storage.
- Experience with BI platforms, reporting tools, data visualization products, ETL engines.
- Experience with data streaming frameworks.
- DevOps experience with a good understanding of continuous delivery and deployment patterns and tools (Jenkins, Artifactory, Maven, etc)
- Experience with Hbase.
- Experience in data management best practices, real-time and batch data integration, and data rationalization
What do you get in return?
- Competitive total rewards package
- Flexible work environment: Why commute? Work remotely from your home, there’s no daily travel requirement to the office!
- Outstanding people: Collaborate with the industry’s top minds.
- Substantial training allowance: Hone your skills or learn new ones; participate in professional development days, attend conferences, become certified, whatever you like!
- Amazing time off: Start with a minimum 3 weeks vacation, 7 sick days, and 2 professional development days!
- Office Allowance: A device of your choosing and personalise your work environment!
- Fun, fun, fun: Blog during work hours; take a day off and volunteer for your favorite charity.
Established in 1997, Pythian is a global IT Data company based in Ottawa, Canada that specializes in designing, implementing, and managing systems that directly contribute to revenue and business success. We help companies adopt disruptive technologies such as advanced analytics, big data, cloud, databases, DevOps and infrastructure management to advance innovation and increase agility. We are focusing on product development leveraging new and exciting technology that will empower our customers through digital transformation. Pythian is seeking a Big Data developer who will be a technical contributor to Pythian's KickAaas product. More than just a job we hire people who love what they do!
Pythian’s Analytics As A Service KickAaaS puts the power of data in the hands of everyone, so they can make more informed decisions, develop new revenue streams, improve customer experiences, and more! This fully-managed, end-to-end service brings your multi-source, multi-format data together in the cloud, then Pythian helps with advanced analytics and visualizations to ensure business users and data scientists alike get what they need from the data.
Intrigued to see what a job is like at Pythian or check us out @Pythian and #pythianlife. Follow @PythianJobs on Twitter and @loveyourdata on Instagram!
Not the right job for you? Check out what other great jobs Pythian has open in Ottawa and around the world! Pythian Careers
- Disclaimers Pythian is an equal opportunity employer and welcomes applications from people with disabilities.
- Accommodations are available on request for candidates taking part in all aspects of the selection process.
- All applicants will need to fulfill the requirements necessary to obtain a background check
- This role requires both that your place of residence is currently in the USA or Canada and that you are legally eligible to work full time in the USA or Canada. At this time, Pythian is unable to accommodate any requests for relocation or sponsorship of individuals’ VISA / work permit applications.
- No recruitment agencies please