How to run a python script in a docker container in jenkins pipeline

Question

I have a docker image in dockerhub that I want to add as an agent in my jenkins pipeline script. As a part of the image, I perform git clone to fetch a repository from github that has multiple python scripts which corresponds to multiple stages in my jenkins pipeline. I tried searching everywhere but I'm not able to find relevant information that talks about how to access the files inside a docker container in jenkins pipeline.

I'm running jenkins on a VM and it has docker installed. The pipeline performs a build on a docker container. Since there are many steps involved in every single stage of the pipeline, I tried to use python API as much as possible.

This is how my dockerfile looks like and the image builds successfully and I'm able to host it in dockerhub. When I run the container, I'm able to see "jenkins_pipeline_scripts" directory which contains all the necessary python scripts for the pipeline stages.

FROM ros:melodic-ros-core-stretch

RUN apt-get update && apt-get -y install python-pip

RUN git clone <private-repo with token>

This is how my current jenkins pipeline script looks like.

pipeline {
    agent {
        docker {
            image '<image name>'
            registryUrl 'https://registry.hub.docker.com'
            registryCredentialsId 'docker-credentials'
            args '--network host -u root:root'
        }
    }

    stages {
        stage('Test') {
            steps {
                sh 'python jenkins_pipeline_scripts/scripts/test.py'   
            }
        }
    }
}

This is the error I'm getting when I execute the job.

$ docker top 05587cd75db5c4282b86b2f1ded2c43a0f4eae161d6c7d7c03d065b0d45e1 -eo pid,comm
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] sh
+ python jenkins_pipeline_scripts/scripts/test.py
python: can't open file 'jenkins_pipeline_scripts/scripts/test.py': [Errno 2] No such file or directory

Please give Jenkins's all console output

yong
– yong

2019-02-21 01:47:11 +00:00
Commented Feb 21, 2019 at 1:47 — yong
– yong, Commented Feb 21, 2019 at 1:47

yong · Accepted Answer · 2019-02-21 02:16:23Z

2

When Jenkins Pipeline to launch the agent container, it will change the container's WORKDIR via -w option and mount the Jenkins job's workspace folder via -v option.

As a result of both options, the Jenkins job's workspace folder will becomes your container's WORKDIR.

Following is my jenkins job console output:

docker run -t -d -u 29001:100 
-w /bld/workspace/test/agent-poc 
-v /bld/workspace/test/agent-poc:/bld/workspace/test/agent-poc:rw,z 
-v /bld/workspace/test/agent-poc@tmp:/bld/workspace/test/agent-poc@tmp:rw,z 
-e ******** -e ******** -e ******** -e ******** 
docker.hub.com/busybox cat

You clone the code when build the image and they are not inside the WORKDIR, thus reports no such file error.

Two approaches to fix your issue.

1) cd your code folder at firstly, you should know that path.

    stage('Test') {
        steps {
            sh '''
              cd <your code folder in container>
              python jenkins_pipeline_scripts/scripts/test.py
            '''   
        }
    }

2) move git clone code repo from Dockerfile into pipeline stage

As I explained at begin, your job's workspace will become container's WORKDIR, thus you can clone your code into jenkins job workspace via pipeline step, then you no need to cd <your code folder in container>.

answered Feb 21, 2019 at 2:16

yong

13.8k1 gold badge19 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Venkatavaradhan Over a year ago

Thank you. I will try this !

Collectives™ on Stack Overflow

How to run a python script in a docker container in jenkins pipeline

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related