LinkedIn - Gradle Plugin for Apache Hadoop an Open-Source Project


On the 13th of August Alex Bain, Senior Software Engineer at LinkedIn, announced their company’s Gradle Plugin will become an open-source project for Apache Hadoop. Furthermore, the Gradle DSL for Apache Hadoop is included in the open-source project.

Many IT organizations have trouble deploying Hadoop in their production environment. That is why the Hadoop Dev Team at LinkedIn adopted Gradle as their primary system for deploying Hadoop. The Hadoop Team found it was easier to extend the build system in Hadoop by defining plugins. With the help of the Hadoop Plugin, the developers at LinkedIn can effectively deploy, test, and build Hadoop applications. The new Hadoop DSL Plugin uses a domain-specific language that makes it easy to specify jobs and workflows in the workflow manager of Azkaban or Apache Oozie.

The Hadoop Plugin contains different tasks that will help developers to facilitate their work with the different Hadoop application frameworks. Many IT developers have found that not all the tools they use in Hadoop will work with their different applications. Developers use different application frameworks to write their different Hadoop jobs. Therefore, it makes it difficult to organize all your Hadoop projects. However, the new Hadoop Plugin makes it easy for developers to organize their different projects regardless of the tool they have used for the project.

LinkedIn Wrote The DSL Plugin Using Groovy

The Groovy language is a derivative of Java. This helps developers at LinkedIn to develop and invoke multiple application frameworks that run at the top of Hadoop. Furthermore, the Groovy DSL helps shield developers from being forced into creating Azkaban or Oozie workflow files.

Any company that writes Hadoop jobs using Gradle should consider downloading the plugin and using it. LinkedIn designed the plugin to save you time and energy when creating and developing your Hadoop workflows. If you’re interested in obtaining your free Hadoop Plugin, please download the plugin at Github.

LinkedIn is offering a free reference guide and examples for developers. Please visit Hadoop DSL Language Reference page to learn more. Furthermore, LinkedIn welcomes all contributions from developers. They hope that developers will send in ideas for documentation enhancement, feedback, bug reports, and pull requests. This is an excellent opportunity for developers of Hadoop to add plugins to their projects to facilitate their work.

Image: flickr.com

Recommended Posts | IT News

IBM Acquires Cognitive Computing Startup AlchemyAP

IBM Acquires Cognitive Computing Startup AlchemyAPI

IBM has acquired Denver-based AlchemyAPI, a cognitive computing company that delivers a variety of text and image analysis capabilities via API. This is IBM's second major acquisition in the field of software analytics after it bought AI startup Cognea, maker of virtual assistants for mobile devices ...
Google Inc. On An Acquisition Spree

Google Inc. On An Acquisition Spree

The Mountain View internet giant was reported to have spent approximately $17 billion on acquisition of such firms in the last two years. This figure beats the amount top rivals including Apple and Microsoft spent on acquisition combined. Google's latest addition to its portfolio ...
Linux Hitting Car Industry

Linux Hitting Car Industry

Linux is a prime example of open-source development, which has been around for decades, has made big strides recently. It has transported itself from the official platform of geekdom to an enterprise-class operating system that will support some of a today's computing needs ...
Terracotta VPN

Terracotta VPN Helps APT Groups Launch Attacks Around the World

Terracotta is a VPN network of over 1500 VPN nodes around the world. The Chinese based VPN service provider has obtained VPN nodes illegally. The company has hacked into unsuspecting companies that have inadequately protected their Windows servers. Each day new VPN nodes are added to the Terracotta ...