By accepting you will be accessing a service provided by a third-party external to https://techjoomla.com/

5 minutes reading time (1062 words)

Packaging system for Joomla extensions

packagingsystemforjoomlaextensions

Joomla extension package is a combination of core, sub-extension, libraries, layouts, core folder, etc.

Many of Techjoomla’s extensions are shipped with other Techjoomla extensions and infrastructure extensions. For eg:-JTicketing is shipped with JLike along with TJ Vendor, TJ Notifications and TJ Reports. Let's assume that each extension code is maintained in separate GitHub repositories.

Now, when it comes to package building it's a tedious task to remember the branch name of each extension, acquire the latest code from different repositories and then build the package manually.

No more worries as we have written a script which will make your life easier.

Here are some prerequisites you need to know before creating the package.

Prerequisites

  • One should have knowledge of 
    • Jenkins CI/CD
    • Jenkins pipeline script and
    • GitHub.

Let’s look at the example of Techjoomla JGive(It’s our own crowdfunding extension) package.

JGive package contains

  • Layouts
  • Libraries
  • Modules
  • Plugins
  • Extensions
    • com_jgive.zip
    • com_activitystream.zip etc.

Here libraries, plugins, and sub extensions have their own GitHub repository.

A. package.json: 

Create package.json file, this file will tell the script source of extension, libraries, plugins, etc and destination in the package.

Path: your_github_repo/build/package.json

{

    "package_name": "name_of_the_package",

    "core_files": {

        "src": "source of the package core files",

        "dest": "builds/(No change here)"

    },

    "subextensions": [ // List of subextensons: Components, libraries, plugins etc.

        {

            "name": "Github repository name",

            "repoUrl": "Github repository clone url",

            "src": "Source code folder path",

            "dest": "builds/destination folder path",

            "zip_name": "optional: if sub extension needs to be compressed",

            "format": "zip/folder ('zip': if file to be compressed otherwise use 'folder')"

            "include/exclude":[  

                "Folder to include/exclude"

            ]

        }

    ],

    "versions": {

        "1.0.0 (Package version number)": {

            "Github repository name(must be exactly same used above in subextensions)": {

                "version": "Github Tag",

                "branch": "Github branch name"

            }

        }

    }

}

 

package_name: Name of the package

core_files:  The core files. E.g layout, modules that are maintained in the same repository of the core component.

sub extensions: List of extensions contained in the package.

versions: This gives us the flexibility to build the package for any version. Version object should have version property for each version and extension branch.

E.g

{

    "package_name": "pkg_jgive",

    "core_files": {

        "src": "./jgive/src/*",

        "dest": "builds/"

    },

    "subextensions": [

        {

            "name": "jgive",

            "repoUrl": "git@github.com:techjoomla/jgive.git",

            "src": "./jgive/src/com_jgive/*",

            "dest": "builds/com_jgive",

            "zip_name": "com_jgive",

            "format": "zip"

        },

        {

            "name": "plg_tjassetsloader",

            "repoUrl": "git@github.com:techjoomla/plg_tjassetsloader.git",

            "src": "./plg_tjassetsloader/src/*",

            "dest": "builds/plugins/system/tjassetsloader",

            "format": "folder"

        },

        {

            "name": "joomla-payments",

            "repoUrl": "git@github.com:techjoomla/joomla-payments.git",

            "src": "joomla-payments/code/plugins/",

            "dest": "builds/plugins/payment/",

            "format": "folder",

            "include": [

                "2checkout",

                "alphauserpoints",

                "authorizenet",

                "bycheck",

                "byorder",

                "jomsocialpoints",

                "paypal"

            ]

        },

        {

            "name": "com_activitystream",

            "repoUrl": "git@github.com:techjoomla/com_activitystream.git",

            "src": "./com_activitystream/src/*",

            "dest": "builds/packages/com_activitystream",

            "zip_name": "com_activitystream",

            "format": "zip"

        }

    ],

    "versions": {

        "2.3.0": {

            "jgive": {

                "branch": "release-2.3.0"

            },

            "plg_tjassetsloader": {

                "branch": "master"

            },

            "joomla-payments": {

                "branch": "release-1.0"

            },

            "com_activitystream": {

                "branch": “release-3.0"

            }

        }

}

B. package.groovy

Create a package.groovy file, just update the version value whichever you want to build and paste the code as it is. 

path: your_github_repo/build/pipelines/package.groovy

//!/usr/bin/env groovy

// Get / Set release name

def  version = '2.3.0'

echo version



pipeline {

    agent any

    stages {

        stage('Cleanup') {

            steps {

                script {

                    // Cleanup previous stuff

                    sh("rm -rf scm")

                    sh("rm -rf builds")



                    // Cleanup jlike git folder, files

                    sh("rm -rf .git")

                    sh("rm -rf .gitlab/merge_request_templates")

                    sh("rm -rf build")



                    // Make directories needed to generate build

                    sh("mkdir builds")

                    sh("mkdir scm")

                }

            }

        }



        stage('Checkout') {

            steps {

                script {

                    // This is important, we need clone into different folder here,

                    // Because, as part of tag based pull, we will be cloning same repo again

                    dir('scm') {

                        checkout scm

                    }

                }

            }

        }



        stage('Cleanup-repos') {

            steps {

                script {

                    def props = readJSON file: 'scm/build/package.json'



                    props['subextensions'].eachWithIndex { item, index ->

                       // cleaup subextensions

                       sh("rm -rf $item.name")

                    }



                }

            }

        }



        stage('Init') {

            steps {

                script {

                    def props = readJSON file: 'scm/build/package.json'



                    // Do clone all subextensions repos by checking out corresponding release branch

                    props['subextensions'].eachWithIndex { item, index ->

                       sh("git clone --branch " + props['versions'][version][item.name]["branch"] + " --depth 1 $item.repoUrl")

                    }

                }

            }

        }



        stage('Copy files & Make zip of subextension') {

            steps {

                script {

                    def props = readJSON file: 'scm/build/package.json'



                    // Copy core files

                    sh("cp -r " + props["core_files"].src + " " + props['core_files'].dest)



                    // Copy Make the zips

                    props['subextensions'].eachWithIndex { item, index ->

                       sh("mkdir -p " + item.dest)



                        if (item.include && item.exclude)

                        {

                            error "Use either include or exclude. Both together not supported!."

                        }



                       def simple_copy = 0



                       if (item.include)

                       {

                            def count = item.include.size()



                            if (count > 0)

                            {

                                def include = item.include.collect { "$item.src/$it" }.join(' ')

                                sh("cp -r $include $item.dest")

                            }

                            else

                            {

                                simple_copy = 1

                            }

                       }

                       else if(item.exclude)

                       {

                           def count = item.exclude.size()



                            if (count > 0)

                            {

                                def exclude = item.exclude.collect { "--exclude $it" }.join(' ')

                                sh("rsync -avz $exclude $item.src $item.dest")

                            }

                            else

                            {

                                simple_copy = 1

                            }

                       }

                       else

                       {

                           simple_copy = 1

                       }



                        if (simple_copy == 1)

                        {

                            sh("cp -r $item.src  $item.dest")

                        }



                        // Create subextension zip and remove folder which is not needed after zip

                       if (item.format == "zip")

                       {

                            sh("cd  $item.dest  && zip -rq ../$item.zip_name" + ".zip .")

                            sh("rm -rf  $item.dest")

                       }

                    }

                }

            }

        }



        stage('Build Package') {

            steps {

                script {

                    // // Get commit id

                    // // @TODO - needs to define shortGitCommit at global level

                    def gitCommit      = ''

                    def shortGitCommit = ''

                    def props = readJSON file: 'scm/build/package.json'



                    // // For branch based build - we need the revision number of tag checked out,

                    // Custom DIR

                    dir('scm') {

                        gitCommit      = sh(returnStdout: true, script: 'git rev-parse HEAD').trim().take(8)

                        shortGitCommit = gitCommit[0..7]

                        echo gitCommit

                        echo shortGitCommit

                    }



                    // Now we are good to create zip for component

                    sh('cd builds && zip -rq ../' + props['package_name'] + '_v' + version + "_" + shortGitCommit + '.zip .')



                    archiveArtifacts props['package_name'] + '_v' + version + "_" + shortGitCommit + '.zip'

                }

            }

        }



        stage('Cleanup folders') {

             steps {

                 script {

                    // Cleanup, so next time we get fresh files

                    sh("rm -r builds")

                 }

             }

         }

    }

}

C. Jenkins Job

Setup the Jenkins job to run the pipeline script, set the script path as build/pipelines/package.groovy, run the job and your package will be ready within a couple of minutes:-).

Do let us know if you are using anything cool to build the package.

0
Simple Tools and Tips To Protect Your Website From...
Using a content delivery network (CDN) for the app...

Related Posts