LinkedIn - Gradle Plugin for Apache Hadoop an Open-Source Project


On the 13th of August Alex Bain, Senior Software Engineer at LinkedIn, announced their company’s Gradle Plugin will become an open-source project for Apache Hadoop. Furthermore, the Gradle DSL for Apache Hadoop is included in the open-source project.

Many IT organizations have trouble deploying Hadoop in their production environment. That is why the Hadoop Dev Team at LinkedIn adopted Gradle as their primary system for deploying Hadoop. The Hadoop Team found it was easier to extend the build system in Hadoop by defining plugins. With the help of the Hadoop Plugin, the developers at LinkedIn can effectively deploy, test, and build Hadoop applications. The new Hadoop DSL Plugin uses a domain-specific language that makes it easy to specify jobs and workflows in the workflow manager of Azkaban or Apache Oozie.

The Hadoop Plugin contains different tasks that will help developers to facilitate their work with the different Hadoop application frameworks. Many IT developers have found that not all the tools they use in Hadoop will work with their different applications. Developers use different application frameworks to write their different Hadoop jobs. Therefore, it makes it difficult to organize all your Hadoop projects. However, the new Hadoop Plugin makes it easy for developers to organize their different projects regardless of the tool they have used for the project.

LinkedIn Wrote the DSL Plugin using Groovy

The Groovy language is a derivative of Java. This helps developers at LinkedIn to develop and invoke multiple application frameworks that run at the top of Hadoop. Furthermore, the Groovy DSL helps shield developers from being forced into creating Azkaban or Oozie workflow files.

Any company that writes Hadoop jobs using Gradle should consider downloading the plugin and using it. LinkedIn designed the plugin to save you time and energy when creating and developing your Hadoop workflows. If you’re interested in obtaining your free Hadoop Plugin, please download the plugin at Github.

LinkedIn is offering a free reference guide and examples for developers. Please visit Hadoop DSL Language Reference page to learn more. Furthermore, LinkedIn welcomes all contributions from developers. They hope that developers will send in ideas for documentation enhancement, feedback, bug reports, and pull requests. This is an excellent opportunity for developers of Hadoop to add plugins to their projects to facilitate their work.

Image: flickr.com

Recommended Posts | IT News

Three Browsers for Windows

From the point of view of a web project developer the news that the browser number is shrinking is a good one. On the other hand, if we give time back to the '90s, when there were only two browsers, we realize that the situation is not exactly ideal.Without going too much into technical details ... ...

Jaguar Land Rover Develops New 'Ghost Car' Safety System

Jaguar Land Rover is developing a revolutionary new SatNat system that will allow drivers a full panoramic view of their surroundings with the added safety feature of a ‘ghost car’ facility. The new system is to be called the, ‘Follow-Me Ghost Car Navigation’ system. It is c ... ...