Maven - Install missing offline dependencies automatically

Sometime it happens that we need to share with someone else part of a complex Java project based on Maven.
The project is probably composed of various modules, with their dependency relationships, but the different modules are also logically separated. So it make completely sense that we'd like to share only on those submodules.

To do that we have to modify the pom.xml and remove the <parent> section and probably also specify properties and dependencies version that are often defined in the super pom.
But with the help of Eclipse and its "Effective POM" view it's not that hard.

The real problem is that all those actions are not enough to have a portable project.
What is missing is the list of dependency defined in the submodules that we do not want to share as source code and that won't be available to be downloaded from Maven public repositories.

Since those artifacts are required to build the project, what we can do is to manually install them from the command line with the typical:

mvn install:install-file 
 -Dfile=my_module.jar 
 -DgroupId=my.private.project 
 -DartifactId=my_module 
 -Dversion=1.0.0 
 -Dpackaging=jar

It works but it contrasts with the idea that our Maven build should be able to do all is required to build our project.

I was looking for an alternative solution and I have found it in this post on Stackoverflow: http://stackoverflow.com/questions/1355548/maven-including-jar-not-found-in-public-repository

 Let me describe it and also offer a ready git repo for a rapid testing of it.

https://github.com/paoloantinori/maven_automated_artifacts_installation

The main idea is to use the exec-maven-plugin to automatically install the dependency for us, replicating the manual activity.
In this way we can script all the manual activities that we needed. To better distinguish between what are the external libraries and our project I suggest a multimodule project:

Parent
|
+---Dependency Project
|
+---Our Interesting Project

Nothing interesting in the parent pom.xml:

  pom
  Sample :: Multimodule
  
  
    main_project
    dependencies_project
  

Let's have a look at the Main Project pom.xml:

  
    
    
      com.example.giallone
      dependencies
      1.0.0
    
    
    
      com.example.giallone
      dep1
      1.0.0
    
    
      com.example.giallone
      dep2
      1.0.0
    
  

As you can see we are defining 2 different things:

Under "real dependencies" you see references to artifacts that the project needs but are not available in Maven Public Repositories.

While instead the first dependency is to trigger the installation of those missing jars, action performed by the other module of this multimodule project.

 If you try now to build only this module it will fail with the following output , since those missing dependencies have not been installed yet:

pantinor@pantinor main_project$ mvn clean install
/usr/lib/jvm/java
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Sample :: Main Project 1.0.0
[INFO] ------------------------------------------------------------------------
Downloading: http://repo.maven.apache.org/maven2/com/example/giallone/dependencies/1.0.0/dependencies-1.0.0.pom
[WARNING] The POM for com.example.giallone:dependencies:jar:1.0.0 is missing, no dependency information available
[WARNING] The POM for com.example.giallone:dep1:jar:1.0.0 is missing, no dependency information available
[WARNING] The POM for com.example.giallone:dep2:jar:1.0.0 is missing, no dependency information available
Downloading: http://repo.maven.apache.org/maven2/com/example/giallone/dependencies/1.0.0/dependencies-1.0.0.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.166s
[INFO] Finished at: Thu Dec 20 13:02:20 GMT 2012
[INFO] Final Memory: 5M/148M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project rules: Could not resolve dependencies for project com.example.giallone:rules:jar:1.0.0: The following artifacts could not be resolved: com.example.giallone:dependencies:jar:1.0.0, com.example.giallone:dep1:jar:1.0.0, com.example.giallone:dep2:jar:1.0.0: Could not find artifact com.example.giallone:dependencies:jar:1.0.0 in central (http://repo.maven.apache.org/maven2) -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

Let's now have a look at the other module, dependencies_project:

  
    
      
        org.codehaus.mojo
        exec-maven-plugin
        1.2.1
        false
        
          
            install-in-repo-dep1
            validate
            
              exec
            
            
              mvn
              
                install:install-file
                -Dfile=${basedir}/lib/dep1.jar
                -DgroupId=com.example.giallone
                -DartifactId=dep1
                -Dversion=1.0.0
                -Dpackaging=jar
              
            
          
          
            install-in-repo-dep2
            validate
            
              exec
            
            
              mvn
              
                install:install-file
                -Dfile=${basedir}/lib/dep2.jar
                -DgroupId=com.example.giallone
                -DartifactId=dep2
                -Dversion=1.0.0
                -Dpackaging=jar
              
            
          
        
      
    
  

What we are doing here is to use exec-maven-plugin to install for us the missing dependencies in the local repo.
Those are the dependencies that prevent our main project to build successfully.

Having configured our main project to be dependent on this one, if we now build our project at the multimodule level, we will be able to trigger the 2 module in the right order:

[INFO] Reactor Build Order:
[INFO] 
[INFO] Sample :: Multimodule
[INFO] Sample :: Dependencies
[INFO] Sample :: Main Project

And when Multimodule Project is built, all the external dependencies are already installed in the previous step, allowing us to see a successful build! 

pantinor@pantinor install_artifacts_sample$ mvn clean install
/usr/lib/jvm/java
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] Sample :: Multimodule
[INFO] Sample :: Dependencies
[INFO] Sample :: Main Project
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Sample :: Multimodule 1.0.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ common ---
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ common ---
[INFO] Installing /data/repositories/github/install_artifacts_sample/pom.xml to /home/pantinor/.m2/repository/com/example/giallone/common/1.0.0/common-1.0.0.pom
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Sample :: Dependencies 1.0.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ dependencies ---
[INFO] Deleting /data/repositories/github/install_artifacts_sample/dependencies_project/target
[INFO] 
[INFO] --- exec-maven-plugin:1.2.1:exec (install-in-repo-dep1) @ dependencies ---
/usr/lib/jvm/java
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Sample :: Dependencies 1.0.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install-file (default-cli) @ dependencies ---
[INFO] Installing /data/repositories/github/install_artifacts_sample/dependencies_project/lib/dep1.jar to /home/pantinor/.m2/repository/com/example/giallone/dep1/1.0.0/dep1-1.0.0.jar
[INFO] Installing /tmp/mvninstall204045843544074291.pom to /home/pantinor/.m2/repository/com/example/giallone/dep1/1.0.0/dep1-1.0.0.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.782s
[INFO] Finished at: Thu Dec 20 13:06:58 GMT 2012
[INFO] Final Memory: 4M/117M
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- exec-maven-plugin:1.2.1:exec (install-in-repo-dep2) @ dependencies ---
/usr/lib/jvm/java
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Sample :: Dependencies 1.0.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install-file (default-cli) @ dependencies ---
[INFO] Installing /data/repositories/github/install_artifacts_sample/dependencies_project/lib/dep2.jar to /home/pantinor/.m2/repository/com/example/giallone/dep2/1.0.0/dep2-1.0.0.jar
[INFO] Installing /tmp/mvninstall3544830050220263281.pom to /home/pantinor/.m2/repository/com/example/giallone/dep2/1.0.0/dep2-1.0.0.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.116s
[INFO] Finished at: Thu Dec 20 13:07:01 GMT 2012
[INFO] Final Memory: 5M/148M
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ dependencies ---
[debug] execute contextualize
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /data/repositories/github/install_artifacts_sample/dependencies_project/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ dependencies ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ dependencies ---
[debug] execute contextualize
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /data/repositories/github/install_artifacts_sample/dependencies_project/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ dependencies ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.10:test (default-test) @ dependencies ---
[INFO] No tests to run.
[INFO] Surefire report directory: /data/repositories/github/install_artifacts_sample/dependencies_project/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- maven-jar-plugin:2.3.2:jar (default-jar) @ dependencies ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: /data/repositories/github/install_artifacts_sample/dependencies_project/target/dependencies-1.0.0.jar
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ dependencies ---
[INFO] Installing /data/repositories/github/install_artifacts_sample/dependencies_project/target/dependencies-1.0.0.jar to /home/pantinor/.m2/repository/com/example/giallone/dependencies/1.0.0/dependencies-1.0.0.jar
[INFO] Installing /data/repositories/github/install_artifacts_sample/dependencies_project/pom.xml to /home/pantinor/.m2/repository/com/example/giallone/dependencies/1.0.0/dependencies-1.0.0.pom
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Sample :: Main Project 1.0.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ rules ---
[INFO] Deleting /data/repositories/github/install_artifacts_sample/main_project/target
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ rules ---
[debug] execute contextualize
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /data/repositories/github/install_artifacts_sample/main_project/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ rules ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ rules ---
[debug] execute contextualize
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /data/repositories/github/install_artifacts_sample/main_project/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ rules ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.10:test (default-test) @ rules ---
[INFO] No tests to run.
[INFO] Surefire report directory: /data/repositories/github/install_artifacts_sample/main_project/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- maven-jar-plugin:2.3.2:jar (default-jar) @ rules ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: /data/repositories/github/install_artifacts_sample/main_project/target/rules-1.0.0.jar
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ rules ---
[INFO] Installing /data/repositories/github/install_artifacts_sample/main_project/target/rules-1.0.0.jar to /home/pantinor/.m2/repository/com/example/giallone/rules/1.0.0/rules-1.0.0.jar
[INFO] Installing /data/repositories/github/install_artifacts_sample/main_project/pom.xml to /home/pantinor/.m2/repository/com/example/giallone/rules/1.0.0/rules-1.0.0.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Sample :: Multimodule ............................. SUCCESS [0.545s]
[INFO] Sample :: Dependencies ............................ SUCCESS [6.800s]
[INFO] Sample :: Main Project ............................ SUCCESS [0.344s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.816s
[INFO] Finished at: Thu Dec 20 13:07:03 GMT 2012
[INFO] Final Memory: 7M/117M
[INFO] ------------------------------------------------------------------------