Menu

  • Home
  • Work
    • Cloud
      • Virtualization
      • IaaS
      • PaaS
    • Java
    • Go
    • C
    • C++
    • JavaScript
    • PHP
    • Python
    • Architecture
    • Others
      • Assembly
      • Ruby
      • Perl
      • Lua
      • Rust
      • XML
      • Network
      • IoT
      • GIS
      • Algorithm
      • AI
      • Math
      • RE
      • Graphic
    • OS
      • Linux
      • Windows
      • Mac OS X
    • BigData
    • Database
      • MySQL
      • Oracle
    • Mobile
      • Android
      • IOS
    • Web
      • HTML
      • CSS
  • Life
    • Cooking
    • Travel
    • Gardening
  • Gallery
  • Video
  • Music
  • Essay
  • Home
  • Work
    • Cloud
      • Virtualization
      • IaaS
      • PaaS
    • Java
    • Go
    • C
    • C++
    • JavaScript
    • PHP
    • Python
    • Architecture
    • Others
      • Assembly
      • Ruby
      • Perl
      • Lua
      • Rust
      • XML
      • Network
      • IoT
      • GIS
      • Algorithm
      • AI
      • Math
      • RE
      • Graphic
    • OS
      • Linux
      • Windows
      • Mac OS X
    • BigData
    • Database
      • MySQL
      • Oracle
    • Mobile
      • Android
      • IOS
    • Web
      • HTML
      • CSS
  • Life
    • Cooking
    • Travel
    • Gardening
  • Gallery
  • Video
  • Music
  • Essay

在Kubernetes中管理和使用Jenkins

20
Jun
2019

在Kubernetes中管理和使用Jenkins

By Alex
/ in Java,PaaS
0 Comments
前言

如何在云原生环境下进行CI/CD,我们先前有一些经验:

  1. 使用Jenkins + Jenkins的Kubernete插件
  2. 在K8S中按需、动态创建执行CI/CD流水线的Agent
  3. 开发Jenkins共享库,简化编写流水线的难度
  4. 为每套环境(development staging production,除了production允许有多套)创建独立的命名空间,对应K8S的namespace和Jenkins的folder
  5. 利用一个特殊的Jenkins Job来复制、创建新的环境

先前我们搭建的云原生平台主要供内部使用,所以不注重产品化。虽然环境可以复制,但是每个Jenkins Job还是依赖于人工通过Jenkins UI创建。

这种人肉管理Job的方式很不方便,现在做的产品化PaaS平台(以下简称平台)也不希望使用Jenkins UI,仅仅想将它作为流水线执行引擎的一种实现。

Jenkins REST API

除了Web UI之外,Jenkins还提供了RESTful风格的API,平台可以通过这些API和Jenkins交互,API的端点路径位于/api。

目前,REST API有三种风格:

  1. XML API,载荷格式为XML,支持XPath查询。API端点路径位于https://ci.gmem.cc/api/xml?...
  2. JSON API,载荷格式JSON,支持JSONP
  3. Python API,载荷格式可以通过 eval(urllib.urlopen("...").read())转换为Python对象

通过REST API,可以创建Jenkins Job、复制Jenkins Job、管理构建队列,以及重启Jenkins服务器。 

API封装

直接调用REST API比较麻烦,以下语言已经有了对API的封装,也就是Jenkins 客户端库:

  1. Ruby:Jenkins API Client
  2. Java:jenkins-rest
  3. Python:JeninsAPI
job-dsl-plugin

使用Jenkins REST API,你的应用程序就可以管理Jenkins服务器,创建Jenkins Job了。但是,社区还有更具K8S风格的解决方案 —— Jenkins Operator,本章先讨论它的依赖job-dsl-plugin。

Jenkins Job DSL / Plugin项目包含两个部分:

  1. 一个DSL,允许用户用Groovy语言来定义一个Job
  2. 一个Jenkins 插件,根据用户定义的上述DSL,生成Jenkins Job

此插件的目的是,尽可能简化Jenkins Job的创建,它允许用户使用编程的方式来配置Job,并且把所有Job的通用元素隐藏在DSL背后。

举例来说,你有个项目,需要单元测试、构建、集成测试、部署四个任务,那么基于DSL只需要编写:

Groovy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
def gitUrl = 'git://github.com/jenkinsci/job-dsl-plugin.git'
 
job('PROJ-unit-tests') {
    scm {
        git(gitUrl)
    }
    triggers {
        scm('*/15 * * * *')
    }
    steps {
        maven('-e clean test')
    }
}
 
job('PROJ-sonar') {
    scm {
        git(gitUrl)
    }
    triggers {
        cron('15 13 * * *')
    }
    steps {
        maven('sonar:sonar')
    }
}
 
job('PROJ-integration-tests') {
    scm {
        git(gitUrl)
    }
    triggers {
        cron('15 1,13 * * *')
    }
    steps {
        maven('-e clean integration-test')
    }
}
 
job('PROJ-release') {
    scm {
        git(gitUrl)
    }
    steps {
        maven('-B release:prepare release:perform')
        shell('cleanup.sh')
    }
}
特性

job-dsl-plugin的特性包括:

  1. 可以通过DSL直接控制XML,因此任何config.xml可以存在的片段都可以通过DSL操控
  2. DSL提供了很多助手方法,方便通用的Job配置,包括scm、trigger、steps
  3. DSL可以直接编写在Job里
  4. DSL可以放在SCM中,并且通过标准的SCM触发机制拉取
基本用法

使用job-dsl-plugin的步骤如下:

  1. 创建一个 Free-style Project类型的Jenkins Job,用于运行DSL,这种Job叫做Seed Job
  2. 配置Seed Job,添加Process Job DSLs类型的Build Step,在其中编写DSL
  3. 运行Seed Job来生成新的、实际有用的Job
DSL

本节列出job-dsl-plugin的常用DSL。完整的DSL API请参考官方文档。

Job

下面的API用于创建不同类型的Job:

Groovy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
// freeStyleJob的别名
job(String name, Closure closure = null)          
 
freeStyleJob(String name, Closure closure = null)
 
buildFlowJob(String name, Closure closure = null)
 
ivyJob(String name, Closure closure = null)      
 
matrixJob(String name, Closure closure = null)    
 
mavenJob(String name, Closure closure = null)    
 
multiJob(String name, Closure closure = null)    
 
workflowJob(String name, Closure closure = null)  
 
multibranchWorkflowJob(String name, Closure closure = null)

上述DSL都返回一个Job对象,并且可以后续继续修改Job: 

Groovy
1
2
3
4
def myJob = freeStyleJob('SimpleJob')
myJob.with {
    description 'A Simple Job'
}
View

下面的API用于创建视图:

Groovy
1
2
3
4
5
6
7
8
9
10
11
12
13
listView(String name, Closure closure = null)            
 
sectionedView(String name, Closure closure = null)      
 
nestedView(String name, Closure closure = null)          
 
deliveryPipelineView(String name, Closure closure = null)
 
buildPipelineView(String name, Closure closure = null)  
 
buildMonitorView(String name, Closure closure = null)    
 
categorizedJobsView(String name, Closure closure = null)
Folder

如果安装(现在的版本默认已经安装)了CloudBees Folders Plugin插件,下面的API可以用于创建目录:

Groovy
1
folder(String name, Closure closure = null)

目录内的对象可以用路径的形式创建:

Groovy
1
2
3
4
5
6
7
folder('project-a')
 
freeStyleJob('project-a/compile')
 
listView('project-a/pipeline')
 
folder('project-a/testing')
Queue

下面的API用于调度一个Job:

Groovy
1
2
queue(String jobName)
queue(Job job)
File IO

下面的API可以用于读取工作区中的文件:

Groovy
1
2
3
InputStream streamFileFromWorkspace(String filePath)
String readFileFromWorkspace(String filePath)
String readFileFromWorkspace(String jobName, String filePath)
Logging

任何DSL脚本都可以访问名为out的变量,调用它可以打印日志到构建日志(Console Output)中:

Groovy
1
out.println('Hello from a Job DSL script!')

如果希望输出信息到Jenkins日志,可以:

Groovy
1
2
3
4
import java.util.logging.Logger
 
Logger logger = Logger.getLogger('org.example.jobdsl')
logger.info('Hello from a Job DSL script!') 
Configure

如果Job DSL不支持某种Option,你可以使用Configure Block来扩展DSL的能力。

Groovy
1
2
3
4
5
6
7
8
job('example') {
    ...
    configure { project ->
        project / buildWrappers / EnvInjectPasswordWrapper {
            injectGlobalPasswords(true)
        }
    }
}

Configure Block允许你对配置文件config.xml(Jenkins的各种对象的配置都放在自己独立目录的config.xml中)进行直接访问。此块的闭包接收到一个groovy.util.Node参数,表示一个XML节点,具体是什么节点取决于Configure Block所在的上下文:

Groovy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
job('example-1') {
    configure { node ->
        // node为project
    }
}
 
mavenJob('example-2') {
    configure { node ->
        // node为maven2-moduleset
    }
}
 
listView('example') {
    configure { node ->
        // node为hudson.model.ListView
    }
}

你可以使用groovy.util.Node的API操控Jenkins对象的配置元素,但是代码会很丑陋。DSL提供了特殊的语法以简化XML操作,需要注意:

  1. 所有查询返回NodeList,因此像操控第一个元素的话需要调用 nodes[0]
  2. 操作符 +用于添加兄弟节点
  3. 不能访问不存在的子元素
  4. 如果元素名和Groovy关键字、Groovy操作符、任何上层上下文对象的属性(例如properties)冲突,则必须使用引号:
    Groovy
    1
    2
    3
    4
    5
    6
    configure {
        //   和上下文对象属性冲突  带有点号
        it / 'properties' / 'com.example.Test' {
            'switch'('on')
        }
    }
  5. 为了简化操作,两个操作符被重写:
    1. / 根据名称查找子节点,如果子节点不存在,则会创建。可以指定子节点属性以匹配第一个带有属性的子节点
    2. <<添加为子节点,右操作数可以是字符串,为子节点名称;也可以是closure,其行为类似于NodeBuilder

下面是一些例子:

Groovy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
// 简单的例子
configure {
    // it是groovy.util.Node对象,表示Job的config.xml的根元素project
    def aNode = it
    // anotherNode是project的子元素
    def anotherNode = aNode / 'blockBuildWhenDownstreamBuilding'
    // 设置此元素的值
    anotherNode.setValue('true')
 
    // 链式调用,注意 / 的操作符优先级很低
    (it / 'blockBuildWhenUpstreamBuilding').setValue('true')
}
 
 
// 添加权限
job('example') {
    configure { project ->
        def matrix = project / 'properties' / 'hudson.security.AuthorizationMatrixProperty' {
            permission('hudson.model.Item.Configure:jill')
            permission('hudson.model.Item.Configure:jack')
        }
        matrix.appendNode('permission', 'hudson.model.Run.Delete:jryan')
    }
}
/*
对应XML:
<project>
    <properties>
        <hudson.security.AuthorizationMatrixProperty>
            <permission>hudson.model.Item.Configure:jill</permission>
            <permission>hudson.model.Item.Configure:jack</permission>
            <permission>hudson.model.Run.Delete:jryan</permission>
        </hudson.security.AuthorizationMatrixProperty>
    </properties>
</project>
对应DSL:
job('example') {
    authorization {
        permission(Permissions.ItemConfigure, 'jill')
        permission(Permissions.ItemConfigure, 'jack')
        permission(Permissions.RunDelete, 'jryan')
    }
}
*/
 
 
// 配置日志轮换插件
job('example') {
    configure { project ->
        // Doesn't take into account existing node
        project << logRotator {
            daysToKeep(-1)
            numToKeep(10)
            artifactDaysToKeep(-1)
            artifactNumToKeep(-1)
        }
 
        // Alters existing value
        (project / logRotator / daysToKeep).value = 2
    }
}
/*
对应XML:
<project>
    <logRotator>
        <daysToKeep>2</daysToKeep>
        <numToKeep>10</numToKeep>
        <artifactDaysToKeep>-1</artifactDaysToKeep>
        <artifactNumToKeep>-1</artifactNumToKeep>
    </logRotator>
</project>
对应DSL:
job('example') {
    logRotator(2, 10, -1, -1)
}
*/
 
 
// 配置电子邮件提醒
def emailTrigger = {
    trigger {
        email {
            recipientList '$PROJECT_DEFAULT_RECIPIENTS'
            subject '$PROJECT_DEFAULT_SUBJECT'
            body '$PROJECT_DEFAULT_CONTENT'
            sendToDevelopers true
            sendToRequester false
            includeCulprits false
            sendToRecipientList true
        }
    }
}
 
job('example') {
    configure { project ->
        project / publishers << 'hudson.plugins.emailext.ExtendedEmailPublisher' {
              recipientList 'Engineering@company.com'
              configuredTriggers {
                  'hudson.plugins.emailext.plugins.trigger.FailureTrigger' emailTrigger
                  'hudson.plugins.emailext.plugins.trigger.FixedTrigger' emailTrigger
              }
              contentType 'default'
              defaultSubject '$DEFAULT_SUBJECT'
              defaultContent '$DEFAULT_CONTENT'
        }
    }
}
/*
对应XML:
<project>
    <publishers>
        <hudson.plugins.emailext.ExtendedEmailPublisher>
            <recipientList>Engineering@company.com</recipientList>
            <configuredTriggers>
                <hudson.plugins.emailext.plugins.trigger.FailureTrigger>
                    <email>
                        <recipientList>$PROJECT_DEFAULT_RECIPIENTS</recipientList>
                        <subject>$PROJECT_DEFAULT_SUBJECT</subject>
                        <body>$PROJECT_DEFAULT_CONTENT</body>
                        <sendToDevelopers>true</sendToDevelopers>
                        <sendToRequester>false</sendToRequester>
                        <includeCulprits>false</includeCulprits>
                        <sendToRecipientList>true</sendToRecipientList>
                    </email>
                </hudson.plugins.emailext.plugins.trigger.FailureTrigger>
                <hudson.plugins.emailext.plugins.trigger.FixedTrigger>
                    <email>
                        <recipientList>$PROJECT_DEFAULT_RECIPIENTS</recipientList>
                        <subject>$PROJECT_DEFAULT_SUBJECT</subject>
                        <body>$PROJECT_DEFAULT_CONTENT</body>
                        <sendToDevelopers>true</sendToDevelopers>
                        <sendToRequester>false</sendToRequester>
                        <includeCulprits>true</includeCulprits>
                        <sendToRecipientList>true</sendToRecipientList>
                    </email>
                </hudson.plugins.emailext.plugins.trigger.FixedTrigger>
            </configuredTriggers>
            <contentType>default</contentType>
            <defaultSubject>$DEFAULT_SUBJECT</defaultSubject>
            <defaultContent>$DEFAULT_CONTENT</defaultContent>
        </hudson.plugins.emailext.ExtendedEmailPublisher>
    </publishers>
</project>
对应DSL:
job('example') {
    publishers {
        extendedEmail('Engineering@company.com') {
            trigger(triggerName: 'Failure', recipientList: '$PROJECT_DEFAULT_RECIPIENTS')
            trigger(triggerName: 'Fixed', recipientList: '$PROJECT_DEFAULT_RECIPIENTS')
        }
    }
}
*/
 
 
// 为Job添加Shell Step
job('example') {
    configure { project ->
        project / builders / 'hudson.tasks.Shell' {
            command 'echo "Hello" > ${WORKSPACE}/out.txt'
        }
    }
}
/*
对应XML:
<project>
    <builders>
        <hudson.tasks.Shell>
            <command>echo "Hello" > ${WORKSPACE}/out.txt</command>
        </hudson.tasks.Shell>
    </builders>
</project>
对应DSL:
job('example') {
    steps {
        shell 'echo "Hello" > ${WORKSPACE}/out.txt'
    }
}
*/
 
 
// 配置Gradle
job('example') {
    configure { project ->
        project / builders << 'hudson.plugins.gradle.Gradle' {
            description ''
            switches '-Dtiming-multiple=5'
            tasks 'test'
            rootBuildScriptDir ''
            buildFile ''
            useWrapper 'true'
            wrapperScript 'gradlew'
        }
    }
}
/*
对应XML:
<project>
    <builders>
        <hudson.plugins.gradle.Gradle>
            <description/>
            <switches>-Dtiming-multiple=5</switches>
            <tasks>test</tasks>
            <rootBuildScriptDir/>
            <buildFile/>
            <useWrapper>true</useWrapper>
            <wrapperScript>gradlew</wrapperScript>
        </hudson.plugins.gradle.Gradle>
    </builders>
</project>
对应DSL:
job('example') {
    steps {
        gradle('test', '-Dtiming-multiple-5', true) {
            it / wrapperScript 'gradlew'
        }
    }
}
*/
 
 
// 配置SVN
job('example') {
    configure { project ->
        project.remove(project / scm) // remove the existing 'scm' element
        project / scm(class: 'hudson.scm.SubversionSCM') {
            locations {
                'hudson.scm.SubversionSCM_-ModuleLocation' {
                    remote 'http://svn.apache.org/repos/asf/tomcat/maven-plugin/trunk'
                    local '.'
                }
            }
            excludedRegions ''
            includedRegions ''
            excludedUsers ''
            excludedRevprop ''
            excludedCommitMessages ''
            workspaceUpdater(class: "hudson.scm.subversion.UpdateUpdater")
        }
    }
}
/*
对应XML:
<project>
    <scm class="hudson.scm.SubversionSCM">
        <locations>
            <hudson.scm.SubversionSCM_-ModuleLocation>
                <remote>http://svn.apache.org/repos/asf/tomcat/maven-plugin/trunk</remote>
                <local>.</local>
            </hudson.scm.SubversionSCM_-ModuleLocation>
        </locations>
        <excludedRegions/>
        <includedRegions/>
        <excludedUsers/>
        <excludedRevprop/>
        <excludedCommitMessages/>
        <workspaceUpdater class="hudson.scm.subversion.UpdateUpdater"/>
    </scm>
</project>
对应DSL:
job('example') {
    scm {
        svn('http://svn.apache.org/repos/asf/tomcat/maven-plugin/trunk')
    }
}
*/
 
 
// 配置GIT
def gitConfigWithSubdir(subdir, remote) {
    { node ->
        // use remote name given
        node / 'userRemoteConfigs' / 'hudson.plugins.git.UserRemoteConfig' / name(remote)
 
        // use local dir given
        node / 'extensions' << 'hudson.plugins.git.extensions.impl.RelativeTargetDirectory' {
            relativeTargetDir subdir
        }
 
        // clean after checkout
        node / 'extensions' << 'hudson.plugins.git.extensions.impl.CleanCheckout'()
    }
}
 
job('example') {
    scm {
        git(
            'git@server:account/repo1.git',
            'remoteB/master',
            gitConfigWithSubdir('repo1', 'remoteB')
        )
    }
}
/*
对应XML:
<project>
    <scm class='hudson.plugins.git.GitSCM'>
        <userRemoteConfigs>
            <hudson.plugins.git.UserRemoteConfig>
                <url>git@server:account/repo1.git</url>
                <name>remoteB</name>
            </hudson.plugins.git.UserRemoteConfig>
        </userRemoteConfigs>
        <branches>
            <hudson.plugins.git.BranchSpec>
                <name>remoteB/master</name>
            </hudson.plugins.git.BranchSpec>
        </branches>
        <extensions>
            <hudson.plugins.git.extensions.impl.RelativeTargetDirectory>
                <relativeTargetDir>repo1</relativeTargetDir>
            </hudson.plugins.git.extensions.impl.RelativeTargetDirectory>
            <hudson.plugins.git.extensions.impl.CleanCheckout/>
        </extensions>
    </scm>
</project>
对应DSL:
job('example') {
    scm {
        git {
            remote {
                name 'remoteB'
                url 'git@server:account/repo1.git'
            }
            extensions {
                relativeTargetDirectory('repo1')
                cleanAfterCheckout()
            }
        }
    }
}
*/
 
 
// 配置Pre-requisite前置步骤
job('example') {
    configure { project ->
        project / builders / 'dk.hlyh.ciplugins.prereqbuildstep.PrereqBuilder' {
            projects('project-A,project-B')
            warningOnly(false)
        }
    }
}
/*
对应XML:
<project>
    <builders>
        <dk.hlyh.ciplugins.prereqbuildstep.PrereqBuilder>
            <projects>project-A,project-B</projects>
            <warningOnly>false</warningOnly>
        </dk.hlyh.ciplugins.prereqbuildstep.PrereqBuilder>
    </builders>
</project>
对应DSL:
job('example') {
    steps {
        prerequisite('project-A, project-B')
    }
}
*/
 
 
// 配置块重用
def switchOn = {
    it / 'properties' / 'com.example.Test' {
        'switch'('on')
    }
}
job('example-1') {
    configure switchOn
}
 
Closure switchOnOrOff(String value) {
    return {
        it / 'properties' / 'com.example.Test' {
            'switch'(value)
        }
    }
}
job('example-1') {
    configure switchOnOrOff('on')
}
 
 
// 根据属性选择子元素、添加孙子元素两个
job('example') {
    configure {
        def scm = it / scm(class: 'org.MyScm')
        scm << 'aChild' {
            serverUrl('http://example.org/product-a')
        }
        scm << 'aChild' {
            serverUrl('http://example.org/product-b')
        }
    }
}
/*
<project>
    <scm class='org.MyScm'>
        <aChild>
            <serverUrl>http://example.org/product-a</serverUrl>
        </aChild>
        <aChild>
            <serverUrl>http://example.org/product-b</serverUrl>
        </aChild>
    </scm>
</project>
*/
__FILE__

使用此变量可以获得被执行的DSL脚本的位置: 

Groovy
1
println("script directory: ${new File(__FILE__).parent.absolutePath}")
SEED_JOB

通过此变量可以访问运行当前DSL的种子任务: 

Groovy
1
2
3
job('example') {
    quietPeriod(SEED_JOB.quietPeriod)
}
JCasC

有了job-dsl-plugin之后,你仍然需要手工登陆到Jenkins UI来创建种子任务。可以考虑安装Jenkins Configuration as Code插件,该插件允许你编写YAML来配置Jenkins。

下面是JCasC配置Jenkins安全的片段:

YAML
1
2
3
4
5
6
7
8
9
10
jenkins:
  securityRealm:
    ldap:
      configurations:
        - groupMembershipStrategy:
            fromUserRecord:
              attributeName: "memberOf"
          inhibitInferRootDN: false
          rootDN: "dc=acme,dc=org"
          server: "ldaps://ldap.acme.org:1636"
起步
  1. 安装Jenkins和本插件
  2. 配置环境变量CASC_JENKINS_CONFIG,它可以是:
    1. 指向包含一系列配置文件的目录,例如/var/jenkins_home/casc_configs
    2. 指向配置文件
    3. 指向URL
  3. 在Manage Jenkins ⇨ Configuration as Code管理本插件
配置

本节给出一个样例,更多的配置示例参考官方文档。

YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
jenkins:
  # 安全配置
  securityRealm:
    ldap:
      configurations:
        - groupMembershipStrategy:
            fromUserRecord:
              attributeName: "memberOf"
          inhibitInferRootDN: false
          rootDN: "dc=acme,dc=org"
          server: "ldaps://ldap.acme.org:1636"
  # 节点配置
  nodes:
    - permanent:
        name: "static-agent"
        remoteFS: "/home/jenkins"
        launcher:
          jnlp:
 
  slaveAgentPort: 50000
  agentProtocols:
    - "jnlp2"
# 开发工具配置
tool:
  git:
    installations:
      - name: git
        home: /usr/local/bin/git
unclassified:
  mailer:
    adminAddress: admin@acme.org
    replyToAddress: do-not-reply@acme.org
    # Note that this does not work right now
    #smtpHost: smtp.acme.org
    smtpPort: 4441
 
# 凭证信息
credentials:
  system:
    domainCredentials:
      credentials:
        - certificate:
            scope: SYSTEM
            id: ssh_private_key
            keyStoreSource:
              fileOnMaster:
                keyStoreFile: /docker/secret/id_rsa
Jenkins Reload

触发Jenkins Reload配置的方式有:

  1. 通过Jenkins UI: Manage Jenkins ⇨ Configuration ⇨ Reload existing configuration
  2. 通过Jenkins CLI
  3. 发送POST请求到JENKINS_URL/configuration-as-code/reload,可能需要凭证信息
kubernetes-operator

这是一个Operator,能够在K8S上对Jenkins进行全面的管理。特性包括:

  1. Pipeline as Code
  2. 支持基于Groovy脚本或者JCasC进行扩展
  3. 安全加固
安装
Operator

首先需要安装CRD:

YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# kubectl apply -f https://raw.githubusercontent.com/jenkinsci/kubernetes-operator/master/deploy/crds/jenkins_v1alpha2_jenkins_crd.yaml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: jenkins.jenkins.io
spec:
  group: jenkins.io
  names:
    kind: Jenkins
    listKind: JenkinsList
    plural: jenkins
    singular: jenkins
  scope: Namespaced
  versions:
    - name : v1alpha2
      served: true
      storage: true
    - name : v1alpha1
      served: true
      storage: false

然后安装此CRD的Operator:

YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
# kubectl apply -f https://raw.githubusercontent.com/jenkinsci/kubernetes-operator/master/deploy/all-in-one-v1alpha2.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkins-operator
spec:
  replicas: 1
  selector:
    matchLabels:
      name: jenkins-operator
  template:
    metadata:
      labels:
        name: jenkins-operator
    spec:
      serviceAccountName: jenkins-operator
      containers:
        - name: jenkins-operator
          image: virtuslab/jenkins-operator:v0.1.0
          command:
          - jenkins-operator
          args: []
          imagePullPolicy: IfNotPresent
          env:
            - name: WATCH_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: OPERATOR_NAME
              value: "jenkins-operator"
Jenkins

要安装一个Jenkins服务,只需要创建Jenkins类型的CR即可,Jenkins Operator会自动完成其它工作:

YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
apiVersion: jenkins.io/v1alpha2
kind: Jenkins
metadata:
  name: example
spec:
  master:
    containers:
    - name: jenkins-master
      image: jenkins/jenkins:lts
      imagePullPolicy: Always
      livenessProbe:
        failureThreshold: 12
        httpGet:
          path: /login
          port: http
          scheme: HTTP
        initialDelaySeconds: 80
        periodSeconds: 10
        successThreshold: 1
        timeoutSeconds: 5
      readinessProbe:
        failureThreshold: 3
        httpGet:
          path: /login
          port: http
          scheme: HTTP
        initialDelaySeconds: 30
        periodSeconds: 10
        successThreshold: 1
        timeoutSeconds: 1
      resources:
        limits:
          cpu: 1500m
          memory: 3Gi
        requests:
          cpu: "1"
          memory: 500Mi
  seedJobs:
  - id: jenkins-operator
    targets: "cicd/jobs/*.jenkins"
    description: "Jenkins Operator repository"
    repositoryBranch: master
    repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git

等Jenkins的Pod可用后,执行下面的命令获取凭证信息:

Shell
1
2
kubectl get secret jenkins-operator-credentials-example -o 'jsonpath={.data.user}' | base64 -d
kubectl get secret jenkins-operator-credentials-example -o 'jsonpath={.data.password}' | base64 -d
编写种子任务和流水线 

Jenkins Operator依赖以下插件:

  1. job-dsl-plugin,使用该插件提供的“种子任务”、DSL
  2.  kubernetes-credentials-provider,在K8S中配置部署密钥

默认情况下Jenkins Operator期望你的项目下有如下目录:

Shell
1
2
3
4
5
cicd/
├── jobs
│   └── build.jenkins
└── pipelines
    └── build.jenkins

其中jobs包含Jenkins Job的定义,基于Job DSL语法:

Groovy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
#!/usr/bin/env groovy
 
// 定义一个流水线任务
pipelineJob('build-jenkins-operator') {
    // 显示名称
    displayName('Build jenkins-operator')
    // 任务定义
    definition {
        cpsScm {
            // 代码库信息
            scm {
                git {
                    remote {
                        url('https://github.com/jenkinsci/kubernetes-operator.git')
                        credentials('jenkins-operator')
                    }
                    branches('*/master')
                }
            }
            // 引用实际构建流水线
            scriptPath('cicd/pipelines/build.jenkins')
        }
    }
}

而pipelines则存放实际的构建流水线,例如: 

Groovy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
#!/usr/bin/env groovy
 
def label = "build-jenkins-operator-${UUID.randomUUID().toString()}"
def home = "/home/jenkins"
def workspace = "${home}/workspace/build-jenkins-operator"
def workdir = "${workspace}/src/github.com/jenkinsci/kubernetes-operator/"
 
// 在此模板所定义的Pod中执行流水线
podTemplate(label: label,
        containers: [
                // 必须有一个名为jnlp的容器,负责和Jenkins服务器的交互
                containerTemplate(name: 'jnlp', image: 'jenkins/jnlp-slave:alpine'),
                // 这个容器则负责运行构建工具
                containerTemplate(name: 'go', image: 'golang:1-alpine', command: 'cat', ttyEnabled: true),
        ],
        envVars: [
                envVar(key: 'GOPATH', value: workspace),
        ],
        ) {
    // 在上述Pod中
    node(label) {
        // 切换工作目录
        dir(workdir) {
            // 然后在go容器中完成构建
            stage('Init') {
                timeout(time: 3, unit: 'MINUTES') {
                    checkout scm
                }
                container('go') {
                    sh 'apk --no-cache --update add make git gcc libc-dev'
                }
            }
 
            stage('Dep') {
                container('go') {
                    sh 'make dep'
                }
            }
 
            stage('Test') {
                container('go') {
                    sh 'make test'
                }
            }
 
            stage('Build') {
                container('go') {
                    sh 'make build'
                }
            }
        }
    }
}
配置种子任务

除了在工程目录中提供上述种子任务、流水线定义,你还需要配置Jenkins CR,告知Operator到什么地方获取种子任务、流水线: 

YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
apiVersion: jenkins.io/v1alpha2
kind: Jenkins
metadata:
  name: example
spec:
  seedJobs:
  - id: jenkins-operator
    targets: "cicd/jobs/*.jenkins"
    description: "Jenkins Operator repository"
    repositoryBranch: master
    repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git
  - id: jenkins-operator-ssh
    # 如果Git仓库是私有的,可以使用基于SSH的身份验证
    credentialType: basicSSHUserPrivateKey
    credentialID: k8s-ssh
    # 也可以使用基于用户名密码的身份验证
  - id: jenkins-operator-user-pass
    credentialType: usernamePassword
    credentialID: k8s-user-pass
 
# 上述k8s-ssh、k8s-user-pass必须配置为K8S Secret:
 
apiVersion: v1
kind: Secret
metadata:
  name: k8s-ssh
data:
  privateKey: |
    -----BEGIN RSA PRIVATE KEY-----
    MIIJKAIBAAKCAgEAxxDpleJjMCN5nusfW/AtBAZhx8UVVlhhhIKXvQ+dFODQIdzO
    oDXybs1zVHWOj31zqbbJnsfsVZ9Uf3p9k6xpJ3WFY9b85WasqTDN1xmSd6swD4N8
    ...
  username: github_user_name
 
 
apiVersion: v1
kind: Secret
metadata:
  name: k8s-user-pass
data:
  username: github_user_name
  password: password_or_token
安装插件

要为Jenkins安装插件,可以配置CR,增加spec.master.plugins字段:

YAML
1
2
3
4
5
6
7
8
9
apiVersion: jenkins.io/v1alpha2
kind: Jenkins
metadata:
  name: example
spec:
  master:
   plugins:
   - name: simple-theme-plugin
     version: 0.5.1
定制Jenkins

定制Jenkins配置,可以使用Groovy脚本或者JCasC。所有定制配置信息都存放在Secret  jenkins-operator-user-configuration-<cr_name>中:

YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# kubectl get configmap jenkins-operator-user-configuration-<cr_name> -o yaml
 
apiVersion: v1
data:
  # 使用Groovy脚本陪孩子
  1-configure-theme.groovy: |2
    import jenkins.*
    import jenkins.model.*
    import hudson.*
    import hudson.model.*
    import org.jenkinsci.plugins.simpletheme.ThemeElement
    import org.jenkinsci.plugins.simpletheme.CssTextThemeElement
    import org.jenkinsci.plugins.simpletheme.CssUrlThemeElement
    
    # 获取Jenkins实例对象
    Jenkins jenkins = Jenkins.getInstance()
 
    # 获取插件配置对象
    def decorator = Jenkins.instance.getDescriptorByType(org.codefirst.SimpleThemeDecorator.class)
 
    List<ThemeElement> configElements = new ArrayList<>();
    configElements.add(new CssTextThemeElement("DEFAULT"));
    configElements.add(new CssUrlThemeElement("https://cdn.rawgit.com/afonsof/jenkins-material-theme/gh-pages/dist/material-light-green.css"));
    decorator.setElements(configElements);
    decorator.save();
    # 保存配置
    jenkins.save()
  # 使用JCasC也可以
  1-system-message.yaml: |2
    jenkins:
      systemMessage: "Configuration as Code integration works!!!"
      adminAddress: "${SECRET_JENKINS_ADMIN_ADDRESS}"
kind: ConfigMap
metadata:
  name: jenkins-operator-user-configuration-<cr_name>
  namespace: default
总结

本文调研的这些技术中,除了Jenkins REST API以外都是面向Jenkins的最终用户的,不适合进行二次开发。

job-dsl-plugin提供的DSL的确具有其价值,它简化了Jenkins API的复杂度,可以避免编写繁琐的XML操控代码。所以,通过Jenkins REST API创建基于job-dsl-plugin的种子任务,并立即触发来生成Jenkins流水线任务是一种可行的方案。

对于最终用户来说,job-dsl-plugin的DSL还是太过复杂,应当进一步简化、屏蔽技术细节。

← Rust学习笔记
如何申请Let's Encrypt免费证书 →

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">

Related Posts

  • Spring Cloud学习笔记
  • SOFAStack学习笔记
  • Axis知识集锦
  • Java线程与并发编程
  • SpringMVC知识集锦

Recent Posts

  • Investigating and Solving the Issue of Failed Certificate Request with ZeroSSL and Cert-Manager
  • A Comprehensive Study of Kotlin for Java Developers
  • 背诵营笔记
  • 利用LangChain和语言模型交互
  • 享学营笔记
ABOUT ME

汪震 | Alex Wong

江苏淮安人,现居北京。目前供职于腾讯云,专注容器方向。

GitHub:gmemcc

Git:git.gmem.cc

Email:gmemjunk@gmem.cc@me.com

ABOUT GMEM

绿色记忆是我的个人网站,域名gmem.cc中G是Green的简写,MEM是Memory的简写,CC则是我的小天使彩彩名字的简写。

我在这里记录自己的工作与生活,同时和大家分享一些编程方面的知识。

GMEM HISTORY
v2.00:微风
v1.03:单车旅行
v1.02:夏日版
v1.01:未完成
v0.10:彩虹天堂
v0.01:阳光海岸
MIRROR INFO
Meta
  • Log in
  • Entries RSS
  • Comments RSS
  • WordPress.org
Recent Posts
  • Investigating and Solving the Issue of Failed Certificate Request with ZeroSSL and Cert-Manager
    In this blog post, I will walk ...
  • A Comprehensive Study of Kotlin for Java Developers
    Introduction Purpose of the Study Understanding the Mo ...
  • 背诵营笔记
    Day 1 Find Your Greatness 原文 Greatness. It’s just ...
  • 利用LangChain和语言模型交互
    LangChain是什么 从名字上可以看出来,LangChain可以用来构建自然语言处理能力的链条。它是一个库 ...
  • 享学营笔记
    Unit 1 At home Lesson 1 In the ...
  • K8S集群跨云迁移
    要将K8S集群从一个云服务商迁移到另外一个,需要解决以下问题: 各种K8S资源的迁移 工作负载所挂载的数 ...
  • Terraform快速参考
    简介 Terraform用于实现基础设施即代码(infrastructure as code)—— 通过代码( ...
  • 草缸2021
    经过四个多月的努力,我的小小荷兰景到达极致了状态。

  • 编写Kubernetes风格的APIServer
    背景 前段时间接到一个需求做一个工具,工具将在K8S中运行。需求很适合用控制器模式实现,很自然的就基于kube ...
  • 记录一次KeyDB缓慢的定位过程
    环境说明 运行环境 这个问题出现在一套搭建在虚拟机上的Kubernetes 1.18集群上。集群有三个节点: ...
  • eBPF学习笔记
    简介 BPF,即Berkeley Packet Filter,是一个古老的网络封包过滤机制。它允许从用户空间注 ...
  • IPVS模式下ClusterIP泄露宿主机端口的问题
    问题 在一个启用了IPVS模式kube-proxy的K8S集群中,运行着一个Docker Registry服务 ...
  • 念爷爷
      今天是爷爷的头七,十二月七日、阴历十月廿三中午,老人家与世长辞。   九月初,回家看望刚动完手术的爸爸,发

  • 6 杨梅坑

  • liuhuashan
    深圳人才公园的网红景点 —— 流花山

  • 1 2020年10月拈花湾

  • 内核缺陷触发的NodePort服务63秒延迟问题
    现象 我们有一个新创建的TKE 1.3.0集群,使用基于Galaxy + Flannel(VXLAN模式)的容 ...
  • Galaxy学习笔记
    简介 Galaxy是TKEStack的一个网络组件,支持为TKE集群提供Overlay/Underlay容器网 ...
TOPLINKS
  • Zitahli's blue 91 people like this
  • 梦中的婚礼 64 people like this
  • 汪静好 61 people like this
  • 那年我一岁 36 people like this
  • 为了爱 28 people like this
  • 小绿彩 26 people like this
  • 杨梅坑 6 people like this
  • 亚龙湾之旅 1 people like this
  • 汪昌博 people like this
  • 彩虹姐姐的笑脸 24 people like this
  • 2013年11月香山 10 people like this
  • 2013年7月秦皇岛 6 people like this
  • 2013年6月蓟县盘山 5 people like this
  • 2013年2月梅花山 2 people like this
  • 2013年淮阴自贡迎春灯会 3 people like this
  • 2012年镇江金山游 1 people like this
  • 2012年徽杭古道 9 people like this
  • 2011年清明节后扬州行 1 people like this
  • 2008年十一云龙公园 5 people like this
  • 2008年之秋忆 7 people like this
  • 老照片 13 people like this
  • 火一样的六月 16 people like this
  • 发黄的相片 3 people like this
  • Cesium学习笔记 90 people like this
  • IntelliJ IDEA知识集锦 59 people like this
  • 基于Kurento搭建WebRTC服务器 38 people like this
  • Bazel学习笔记 38 people like this
  • PhoneGap学习笔记 32 people like this
  • NaCl学习笔记 32 people like this
  • 使用Oracle Java Mission Control监控JVM运行状态 29 people like this
  • 基于Calico的CNI 27 people like this
  • Ceph学习笔记 27 people like this
  • Three.js学习笔记 24 people like this
Tag Cloud
ActiveMQ AspectJ CDT Ceph Chrome CNI Command Cordova Coroutine CXF Cygwin DNS Docker eBPF Eclipse ExtJS F7 FAQ Groovy Hibernate HTTP IntelliJ IO编程 IPVS JacksonJSON JMS JSON JVM K8S kernel LB libvirt Linux知识 Linux编程 LOG Maven MinGW Mock Monitoring Multimedia MVC MySQL netfs Netty Nginx NIO Node.js NoSQL Oracle PDT PHP Redis RPC Scheduler ServiceMesh SNMP Spring SSL svn Tomcat TSDB Ubuntu WebGL WebRTC WebService WebSocket wxWidgets XDebug XML XPath XRM ZooKeeper 亚龙湾 单元测试 学习笔记 实时处理 并发编程 彩姐 性能剖析 性能调优 文本处理 新特性 架构模式 系统编程 网络编程 视频监控 设计模式 远程调试 配置文件 齐塔莉
Recent Comments
  • qg on Istio中的透明代理问题
  • heao on 基于本地gRPC的Go插件系统
  • 黄豆豆 on Ginkgo学习笔记
  • cloud on OpenStack学习笔记
  • 5dragoncon on Cilium学习笔记
  • Archeb on 重温iptables
  • C/C++编程:WebSocketpp(Linux + Clion + boostAsio) – 源码巴士 on 基于C/C++的WebSocket库
  • jerbin on eBPF学习笔记
  • point on Istio中的透明代理问题
  • G on Istio中的透明代理问题
  • 绿色记忆:Go语言单元测试和仿冒 on Ginkgo学习笔记
  • point on Istio中的透明代理问题
  • 【Maven】maven插件开发实战 – IT汇 on Maven插件开发
  • chenlx on eBPF学习笔记
  • Alex on eBPF学习笔记
  • CFC4N on eBPF学习笔记
  • 李运田 on 念爷爷
  • yongman on 记录一次KeyDB缓慢的定位过程
  • Alex on Istio中的透明代理问题
  • will on Istio中的透明代理问题
  • will on Istio中的透明代理问题
  • haolipeng on 基于本地gRPC的Go插件系统
  • 吴杰 on 基于C/C++的WebSocket库
©2005-2025 Gmem.cc | Powered by WordPress | 京ICP备18007345号-2