LinkedIn Makes the Gradle Plugin for Apache Hadoop an Open-Source Project


On the 13th of August Alex Bain, Senior Software Engineer at LinkedIn, announced their company’s Gradle Plugin will become an open-source project for Apache Hadoop. Furthermore, the Gradle DSL for Apache Hadoop is included in the open-source project.

Many IT organizations have trouble deploying Hadoop in their production environment. That is why the Hadoop Dev Team at LinkedIn adopted Gradle as their primary system for deploying Hadoop. The Hadoop Team found it was easier to extend the build system in Hadoop by defining plugins. With the help of the Hadoop Plugin, the developers at LinkedIn can effectively deploy, test, and build Hadoop applications. The new Hadoop DSL Plugin uses a domain-specific language that makes it easy to specify jobs and workflows in the workflow manager of Azkaban or Apache Oozie

The Hadoop Plugin contains different tasks that will help developers to facilitate their work with the different Hadoop application frameworks. Many IT developers have found that not all the tools they use in Hadoop will work with their different applications. Developers use different application frameworks to write their different Hadoop jobs. Therefore, it makes it difficult to organize all your Hadoop projects. However, the new Hadoop Plugin makes it easy for developers to organize their different projects regardless of the tool they have used for the project.  

LinkedIn wrote the DSL Plugin using Groovy. The Groovy language is a derivative of Java. This helps developers at LinkedIn to develop and invoke multiple application frameworks that run at the top of Hadoop. Furthermore, the Groovy DSL helps shield developers from being forced into creating Azkaban or Oozie workflow files.  

Any company that writes Hadoop jobs using Gradle should consider downloading the plugin and using it. LinkedIn designed the plugin to save you time and energy when creating and developing your Hadoop workflows. If you’re interested in obtaining your free Hadoop Plugin, please download the plugin at Github.

LinkedIn is offering a free reference guide and examples for developers. Please visit Hadoop DSL Language Reference page to learn more. Furthermore, LinkedIn welcomes all contributions from developers. They hope that developers will send in ideas for documentation enhancement, feedback, bug reports, and pull requests. This is an excellent opportunity for developers of Hadoop to add plugins to their projects to facilitate their work.

Image: flickr.com

Top Posts | IT News