Thursday, December 20, 2012

Maven - Install missing offline dependencies automatically

Sometime it happens that we need to share with someone else part of a complex Java project based on Maven.
The project is probably composed of various modules, with their dependency relationships, but the different modules are also logically separated. So it make completely sense that we'd like to share only on those submodules.

To do that we have to modify the pom.xml and remove the <parent> section and probably also specify properties and dependencies version that are often defined in the super pom.
But with the help of Eclipse and its "Effective POM" view it's not that hard.

The real problem is that all those actions are not enough to have a portable project.
What is missing is the list of dependency defined in the submodules that we do not want to share as source code and that won't be available to be downloaded from Maven public repositories.

Since those artifacts are required to build the project, what we can do is to manually install them from the command line with the typical:

mvn install:install-file 
 -Dfile=my_module.jar 
 -DgroupId=my.private.project 
 -DartifactId=my_module 
 -Dversion=1.0.0 
 -Dpackaging=jar
It works but it contrasts with the idea that our Maven build should be able to do all is required to build our project.

I was looking for an alternative solution and I have found it in this post on Stackoverflow: http://stackoverflow.com/questions/1355548/maven-including-jar-not-found-in-public-repository

 Let me describe it and also offer a ready git repo for a rapid testing of it.

https://github.com/paoloantinori/maven_automated_artifacts_installation

The main idea is to use the exec-maven-plugin to automatically install the dependency for us, replicating the manual activity.
In this way we can script all the manual activities that we needed. To better distinguish between what are the external libraries and our project I suggest a multimodule project:

Parent
|
+---Dependency Project
|
+---Our Interesting Project
Nothing interesting in the parent pom.xml:

  pom
  Sample :: Multimodule
  
  
    main_project
    dependencies_project
  

Let's have a look at the Main Project pom.xml:
  
    
    
      com.example.giallone
      dependencies
      1.0.0
    
    
    
      com.example.giallone
      dep1
      1.0.0
    
    
      com.example.giallone
      dep2
      1.0.0
    
  
As you can see we are defining 2 different things:

Under "real dependencies" you see references to artifacts that the project needs but are not available in Maven Public Repositories.

While instead the first dependency is to trigger the installation of those missing jars, action performed by the other module of this multimodule project.

 If you try now to build only this module it will fail with the following output , since those missing dependencies have not been installed yet:


pantinor@pantinor main_project$ mvn clean install
/usr/lib/jvm/java
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Sample :: Main Project 1.0.0
[INFO] ------------------------------------------------------------------------
Downloading: http://repo.maven.apache.org/maven2/com/example/giallone/dependencies/1.0.0/dependencies-1.0.0.pom
[WARNING] The POM for com.example.giallone:dependencies:jar:1.0.0 is missing, no dependency information available
[WARNING] The POM for com.example.giallone:dep1:jar:1.0.0 is missing, no dependency information available
[WARNING] The POM for com.example.giallone:dep2:jar:1.0.0 is missing, no dependency information available
Downloading: http://repo.maven.apache.org/maven2/com/example/giallone/dependencies/1.0.0/dependencies-1.0.0.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.166s
[INFO] Finished at: Thu Dec 20 13:02:20 GMT 2012
[INFO] Final Memory: 5M/148M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project rules: Could not resolve dependencies for project com.example.giallone:rules:jar:1.0.0: The following artifacts could not be resolved: com.example.giallone:dependencies:jar:1.0.0, com.example.giallone:dep1:jar:1.0.0, com.example.giallone:dep2:jar:1.0.0: Could not find artifact com.example.giallone:dependencies:jar:1.0.0 in central (http://repo.maven.apache.org/maven2) -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException


Let's now have a look at the other module, dependencies_project:
  
    
      
        org.codehaus.mojo
        exec-maven-plugin
        1.2.1
        false
        
          
            install-in-repo-dep1
            validate
            
              exec
            
            
              mvn
              
                install:install-file
                -Dfile=${basedir}/lib/dep1.jar
                -DgroupId=com.example.giallone
                -DartifactId=dep1
                -Dversion=1.0.0
                -Dpackaging=jar
              
            
          
          
            install-in-repo-dep2
            validate
            
              exec
            
            
              mvn
              
                install:install-file
                -Dfile=${basedir}/lib/dep2.jar
                -DgroupId=com.example.giallone
                -DartifactId=dep2
                -Dversion=1.0.0
                -Dpackaging=jar
              
            
          
        
      
    
  
What we are doing here is to use exec-maven-plugin to install for us the missing dependencies in the local repo.
Those are the dependencies that prevent our main project to build successfully.

Having configured our main project to be dependent on this one, if we now build our project at the multimodule level, we will be able to trigger the 2 module in the right order:

[INFO] Reactor Build Order:
[INFO] 
[INFO] Sample :: Multimodule
[INFO] Sample :: Dependencies
[INFO] Sample :: Main Project


And when Multimodule Project is built, all the external dependencies are already installed in the previous step, allowing us to see a successful build! 

pantinor@pantinor install_artifacts_sample$ mvn clean install
/usr/lib/jvm/java
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] Sample :: Multimodule
[INFO] Sample :: Dependencies
[INFO] Sample :: Main Project
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Sample :: Multimodule 1.0.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ common ---
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ common ---
[INFO] Installing /data/repositories/github/install_artifacts_sample/pom.xml to /home/pantinor/.m2/repository/com/example/giallone/common/1.0.0/common-1.0.0.pom
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Sample :: Dependencies 1.0.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ dependencies ---
[INFO] Deleting /data/repositories/github/install_artifacts_sample/dependencies_project/target
[INFO] 
[INFO] --- exec-maven-plugin:1.2.1:exec (install-in-repo-dep1) @ dependencies ---
/usr/lib/jvm/java
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Sample :: Dependencies 1.0.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install-file (default-cli) @ dependencies ---
[INFO] Installing /data/repositories/github/install_artifacts_sample/dependencies_project/lib/dep1.jar to /home/pantinor/.m2/repository/com/example/giallone/dep1/1.0.0/dep1-1.0.0.jar
[INFO] Installing /tmp/mvninstall204045843544074291.pom to /home/pantinor/.m2/repository/com/example/giallone/dep1/1.0.0/dep1-1.0.0.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.782s
[INFO] Finished at: Thu Dec 20 13:06:58 GMT 2012
[INFO] Final Memory: 4M/117M
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- exec-maven-plugin:1.2.1:exec (install-in-repo-dep2) @ dependencies ---
/usr/lib/jvm/java
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Sample :: Dependencies 1.0.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install-file (default-cli) @ dependencies ---
[INFO] Installing /data/repositories/github/install_artifacts_sample/dependencies_project/lib/dep2.jar to /home/pantinor/.m2/repository/com/example/giallone/dep2/1.0.0/dep2-1.0.0.jar
[INFO] Installing /tmp/mvninstall3544830050220263281.pom to /home/pantinor/.m2/repository/com/example/giallone/dep2/1.0.0/dep2-1.0.0.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.116s
[INFO] Finished at: Thu Dec 20 13:07:01 GMT 2012
[INFO] Final Memory: 5M/148M
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ dependencies ---
[debug] execute contextualize
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /data/repositories/github/install_artifacts_sample/dependencies_project/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ dependencies ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ dependencies ---
[debug] execute contextualize
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /data/repositories/github/install_artifacts_sample/dependencies_project/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ dependencies ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.10:test (default-test) @ dependencies ---
[INFO] No tests to run.
[INFO] Surefire report directory: /data/repositories/github/install_artifacts_sample/dependencies_project/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- maven-jar-plugin:2.3.2:jar (default-jar) @ dependencies ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: /data/repositories/github/install_artifacts_sample/dependencies_project/target/dependencies-1.0.0.jar
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ dependencies ---
[INFO] Installing /data/repositories/github/install_artifacts_sample/dependencies_project/target/dependencies-1.0.0.jar to /home/pantinor/.m2/repository/com/example/giallone/dependencies/1.0.0/dependencies-1.0.0.jar
[INFO] Installing /data/repositories/github/install_artifacts_sample/dependencies_project/pom.xml to /home/pantinor/.m2/repository/com/example/giallone/dependencies/1.0.0/dependencies-1.0.0.pom
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Sample :: Main Project 1.0.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ rules ---
[INFO] Deleting /data/repositories/github/install_artifacts_sample/main_project/target
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ rules ---
[debug] execute contextualize
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /data/repositories/github/install_artifacts_sample/main_project/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ rules ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ rules ---
[debug] execute contextualize
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /data/repositories/github/install_artifacts_sample/main_project/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ rules ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.10:test (default-test) @ rules ---
[INFO] No tests to run.
[INFO] Surefire report directory: /data/repositories/github/install_artifacts_sample/main_project/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- maven-jar-plugin:2.3.2:jar (default-jar) @ rules ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: /data/repositories/github/install_artifacts_sample/main_project/target/rules-1.0.0.jar
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ rules ---
[INFO] Installing /data/repositories/github/install_artifacts_sample/main_project/target/rules-1.0.0.jar to /home/pantinor/.m2/repository/com/example/giallone/rules/1.0.0/rules-1.0.0.jar
[INFO] Installing /data/repositories/github/install_artifacts_sample/main_project/pom.xml to /home/pantinor/.m2/repository/com/example/giallone/rules/1.0.0/rules-1.0.0.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Sample :: Multimodule ............................. SUCCESS [0.545s]
[INFO] Sample :: Dependencies ............................ SUCCESS [6.800s]
[INFO] Sample :: Main Project ............................ SUCCESS [0.344s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.816s
[INFO] Finished at: Thu Dec 20 13:07:03 GMT 2012
[INFO] Final Memory: 7M/117M
[INFO] ------------------------------------------------------------------------

23 comments:

  1. Hey, great post, it helped a lot. As a token of gratitude i'll contribute with more knowledge that can help improve this document:
    You can use the maven install plugin from the maven pom directly without having to re-invoke maven. Here is how you can do it:

    org.apache.maven.plugins
    maven-install-plugin
    2.4

    relative/path/to/file.jar
    pt.example.package
    package
    version
    jar



    validate

    install-file



    ReplyDelete
    Replies
    1. the xml tags got screwed up, here's the code:
      <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-install-plugin</artifactId>
      <version>2.4</version>
      <configuration>
      <file>relative/path/to/file.jar</file>
      <groupId>pt.example.package</groupId>
      <artifactId>package</artifactId>
      <version>version</version>
      <packaging>jar</packaging>
      </configuration>
      <executions>
      <execution>
      <phase>validate</phase>
      <goals>
      <goal>install-file</goal>
      </goals>
      </execution>
      </executions>
      </plugin>

      Delete
  2. you can download all the dependecies here: http://jar-download.com/

    ReplyDelete
  3. you can download all the dependecies here: http://jar-download.com/

    ReplyDelete
  4. thanks for Providing a Good Information
    anyone want to learn advance devops tools or devops online training visit:
    DevOps Training
    DevOps Online Training

    ReplyDelete
  5. Publish your printed book to Kindle or any other hard copy and paperback books to Kindle Mobi services with Ebookconversion service solutions.
    Hard Copy to Kindle Conversion

    ReplyDelete

  6. Dr. Vivek Galani is a leading expert in skin and hair. At hair transplant clinic in Surat Skin Care, Cosmetic Laser, Hair Transplant & Slimming Center, Dr. Galani offers the most advanced cosmetic and dermatologic care treatments. The clinic uses advanced FUE methods to produce high-quality hair transplants.

    ReplyDelete
  7. Thanks for Sharing This Article. It is very so much valuable content.
    DevOps Training
    DevOps Online Training

    ReplyDelete

  8. Hey! Lovely blog. Your blog contains all the details and information related to the topic. In case you are a QuickBooks user, here is good news for you. You may encounter any error like QuickBooks Error, visit at QuickBooks Customer Service for quick help.

    ReplyDelete
  9. We SVJ Technocoat are the leading Service Provider and Exporter of an extensive array of PVD Coating In Surat Service and Vapor Deposition Coating Service etc. We are a well known firm for providing excellent quality coating services across the nation and in a timely manner. Owing to our improvised business models, our professionals are offering integrated solutions for our clients

    ReplyDelete
  10. Such a nice blog, thanks for sharing with everyone, if you face trouble resolving QuickBooks Errors, Contact:Quickbooks customer service number.+18554442233

    ReplyDelete
  11. nice blog. you are looking for a best custumer service Quickbooks support serviceyou can contact us at.+13464148256

    ReplyDelete
  12. This comment has been removed by the author.

    ReplyDelete
  13. We understand how important it is to mark your business presence online. Over the years, we have gained strong domain experience and expertise in technologies for web, mobile and software design and development.

    software development company in surat

    ReplyDelete
  14. We are a manufacturer of Organic stevia drops (sugar alternatives) in Surat & Supply Across India. It Is Ideal For Diabetics, weight loss, etc... Buy Now!

    stevia for diabetic patients

    ReplyDelete
  15. CP International is one of the Leading Exporters, Manufacturers & Suppliers of Indian Spices, Pooja items, Homemade & Dry fruits Products, etc. Across USA & UK.

    Leading Indian Spices Exporters in USA

    ReplyDelete
  16. Nice article thank you for the sharing article. If you are irritated with your blocking website then I have one article for you unblocked memes this article will help you access your premises' restricted or censored content. your everyday schedule likely will permit you to download it or even visit the site.

    ReplyDelete
  17. Use Bulk whatsApp Sender to Reach your customers at Lighting Speed with Whatso.
    Features:
    Grab Contacts from WhatsApp Groups, Fast sending mode, Schedule Sending & Numbers Filter.

    ReplyDelete