LinkedIn - Gradle Plugin for Apache Hadoop an Open-Source Project


On the 13th of August Alex Bain, Senior Software Engineer at LinkedIn, announced their company’s Gradle Plugin will become an open-source project for Apache Hadoop. Furthermore, the Gradle DSL for Apache Hadoop is included in the open-source project.

Many IT organizations have trouble deploying Hadoop in their production environment. That is why the Hadoop Dev Team at LinkedIn adopted Gradle as their primary system for deploying Hadoop. The Hadoop Team found it was easier to extend the build system in Hadoop by defining plugins. With the help of the Hadoop Plugin, the developers at LinkedIn can effectively deploy, test, and build Hadoop applications. The new Hadoop DSL Plugin uses a domain-specific language that makes it easy to specify jobs and workflows in the workflow manager of Azkaban or Apache Oozie.

The Hadoop Plugin contains different tasks that will help developers to facilitate their work with the different Hadoop application frameworks. Many IT developers have found that not all the tools they use in Hadoop will work with their different applications. Developers use different application frameworks to write their different Hadoop jobs. Therefore, it makes it difficult to organize all your Hadoop projects. However, the new Hadoop Plugin makes it easy for developers to organize their different projects regardless of the tool they have used for the project.

LinkedIn Wrote The DSL Plugin Using Groovy

The Groovy language is a derivative of Java. This helps developers at LinkedIn to develop and invoke multiple application frameworks that run at the top of Hadoop. Furthermore, the Groovy DSL helps shield developers from being forced into creating Azkaban or Oozie workflow files.

Any company that writes Hadoop jobs using Gradle should consider downloading the plugin and using it. LinkedIn designed the plugin to save you time and energy when creating and developing your Hadoop workflows. If you’re interested in obtaining your free Hadoop Plugin, please download the plugin at Github.

LinkedIn is offering a free reference guide and examples for developers. Please visit Hadoop DSL Language Reference page to learn more. Furthermore, LinkedIn welcomes all contributions from developers. They hope that developers will send in ideas for documentation enhancement, feedback, bug reports, and pull requests. This is an excellent opportunity for developers of Hadoop to add plugins to their projects to facilitate their work.

Image: flickr.com

Recommended Posts | IT News

Facial Recognition

New Innovations in Facial Recognition, Software To Aid And Law

The new facial recognition software strikes fear among citizens across the world, while law enforcement and business leaders tout the advantages. Many people in the private sector are voicing concerns about invasion of privacy. Law enforcement officials are excited over the prospects of apprehending ...
Google Nest’s Acquires Revolv

Google Nest’s Acquires Revolv

Google has bought Revolv, a startup company in home automation, to help Google's subdivision known as Nest. Nest labs is a digital thermostat company, which was acquired by Google recently. These acquisitions are geared towards Google's investment in appliances for smarter homes ...
Google Aims To Boost Cloud Tools By Buying Zync

Google Aims To Boost Cloud Tools By Buying Zync

Google bought Zync Inc. to bolster its cloud-based offerings by including Zync's technologies to its services.Zync is a Boston-based visual-effects software company, whose Zync Render technology is used for TV and film studios, and was used in developing special effects for movies ...
Apple Acquires FoundationDB

Apple Acquires FoundationDB To Improve Cloud-based Services

Apple acquired startup that specializes in speedy, durable NoSQL database. Apple may improve its own cloud-based services. Cupertino-based Apple has agreed to acquire FoundationDB, a Virginia-based database company that develops scalable and fault tolerant databases that support multiple data models ...