In the rapidly evolving field of software development, staying abreast of emerging technologies is essential for maintaining a competitive edge. Kotlin, a statically typed programming language developed by JetBrains, has garnered significant attention since its release. For developers proficient in Java—especially versions 8 and earlier—exploring Kotlin offers an opportunity to enhance coding efficiency, embrace modern programming paradigms, and address some of the limitations inherent in older versions of Java.

The motivations for learning Kotlin are multifaceted:

1. Modern Language Features: Kotlin introduces contemporary features such as null safety, data classes, and coroutines, which streamline coding practices and reduce common programming errors.
2. Interoperability: Kotlin is fully interoperable with Java, allowing for seamless integration into existing Java projects and the use of established Java libraries.
3. Industry Adoption: Major companies, including Google, have endorsed Kotlin for Android development, signaling a shift in industry standards and practices.

As a Java developer delving into Kotlin, it's important to set clear, achievable goals to maximize the benefits of this study:

1. Comprehensive Understanding: Gain a thorough grasp of Kotlin's syntax, features, and best practices.
2. Comparative Analysis: Identify and understand the similarities and differences between Kotlin and Java to leverage existing knowledge effectively.
3. Practical Application: Apply Kotlin concepts in real-world scenarios, including Android development and server-side applications.
4. Interoperability Proficiency: Learn how to integrate Kotlin code with Java, enabling the use of both languages within the same project seamlessly.
5. Code Optimization: Utilize Kotlin's features to write more concise, efficient, and maintainable code compared to traditional Java approaches.

By setting these goals, the study aims to provide a structured pathway to not only learn Kotlin but also to enhance overall programming proficiency.

Java has been a cornerstone in the software development industry since its inception by Sun Microsystems in 1995. Over the years, it has undergone significant transformations, introducing features that address the evolving needs of developers and the industry at large. Below is a chronological overview of Java's evolution up to Java 21, the latest version as of October 2023.

Java 1.0 to Java 1.4 (1996 - 2002): Establishing the Foundation

1. Java 1.0 (1996): The initial release provided the basic framework of the language, focusing on portability and network computing.
2. Java 1.1 to 1.4: These versions introduced inner classes, JDBC, JavaBeans, the Collections Framework, and enhanced performance and security features.

Java 5 (2004): Embracing Modern Programming Concepts

1. Generics: Allowed for type-safe collections.
2. Annotations: Provided metadata that could be processed by the compiler or at runtime.
3. Enhanced for-loop: Simplified iteration over collections and arrays.
4. Autoboxing/Unboxing: Automated conversion between primitive types and their corresponding object wrapper classes.

Java 6 and Java 7 (2006 - 2011): Performance and Usability Enhancements

1. Java 6: Focused on performance improvements and included updates to the JVM and core libraries.

1. Java 7:
   1. Diamond Operator (<>): Simplified the use of generics.
   2. Try-with-Resources: Enhanced exception handling and resource management.
   3. Strings in Switch Statements: Allowed strings to be used in switch cases.

Java 8 (2014): A Paradigm Shift with Functional Programming

1. Lambda Expressions: Introduced functional programming concepts, enabling more concise code.
2. Stream API: Provided a powerful way to process collections in a functional style.
3. Optional Class: Addressed null references by providing a container object which may or may not contain a non-null value.
4. Date and Time API: Offered a new set of classes under java.time package for date and time manipulation.

Java 9 (2017): Modularization and JShell

1. Project Jigsaw (Modules): Introduced the Java Platform Module System, allowing for better encapsulation and modularization of code.
2. JShell (REPL): Provided an interactive Read-Eval-Print Loop tool for rapid prototyping.

Java 10 (2018): Local Variable Type Inference

1. var Keyword: Enabled local variable type inference, allowing the compiler to infer the type of a variable from its initializer.

Java 11 (2018): Long-Term Support and New Features

1. Standardized HTTP Client API: Introduced a new HTTP client under java.net.http.
2. String Methods Enhancements: Added methods like isBlank(), lines(), strip(), repeat().
3. Removal of JavaFX: Decoupled JavaFX from the JDK.

Java 12 to Java 15 (2019 - 2020): Incremental Improvements

1. Java 12: Switch Expressions (Preview): Enhanced switch statements to be used as expressions.
2. Java 13: Text Blocks (Preview): Simplified the inclusion of multi-line strings.
3. Java 14:
   1. Records (Preview): Introduced a compact syntax for declaring data classes.
   2. Helpful NullPointerExceptions: Improved the detail in NullPointerException messages.

Java 15:

1. Sealed Classes (Preview): Restricted which classes can extend or implement a class or interface.
2. Z Garbage Collector (Product Feature): Low-latency garbage collector moved from experimental to production.

Java 16 and Java 17 (2021): Pattern Matching and Sealed Classes

1. Java 16:
   1. Pattern Matching for instanceof: Simplified the use of instanceof with pattern variables.
   2. Records: Moved from preview to a standard feature.
2. Java 17 (Long-Term Support Release):
   1. Sealed Classes: Finalized as a standard feature.
   2. Removal of Deprecated Features: Eliminated older features like the Applet API.
   3. Enhanced Pseudorandom Number Generators: Introduced new interfaces and implementations for PRNGs.

Java 18 and Java 19 (2022): Incubator and Preview Features

1. Java 18:
   1. UTF-8 by Default: Standardized UTF-8 as the default character set.
   2. Simple Web Server: Provided a command-line tool for starting a minimal web server.
2. Java 19:
   1. Virtual Threads (Preview): Part of Project Loom, introduced lightweight threads for concurrent programming.
   2. Structured Concurrency (Incubator): Simplified multithreaded programming by treating multiple tasks running in different threads as a single unit.

Java 20 and Java 21 (2023): Advancements in Performance and Productivity

1. Java 20:
   1. Scoped Values (Incubator): Allowed for the sharing of immutable data within and across threads.
   2. Record Patterns (Second Preview): Enhanced pattern matching for records.
2. Java 21 (Latest LTS as of October 2023):
   1. Virtual Threads (Standard Feature): Finalized virtual threads for high-throughput concurrent applications.
   2. Sequenced Collections: Introduced interfaces to represent collections with a defined encounter order.
   3. String Templates (Preview): Provided a new way to create and process strings with embedded expressions.
   4. Pattern Matching for Switch (Standard Feature): Finalized pattern matching in switch expressions and statements.

In 2010 JetBrains began the development of Kotlin, aiming to create a language that could improve developer productivity and happiness. The primary motivations were:

1. Conciseness: Reduce boilerplate code common in Java.
2. Safety: Introduce features like null safety to prevent common errors.
3. Interoperability: Ensure seamless integration with Java code and libraries.
4. Tooling Support: Leverage JetBrains' expertise in IDE development to provide excellent tooling from the outset.

In July 2011, JetBrains publicly announced Kotlin, revealing their plans to create a new language for the JVM.

Kotlin 1.0 (February 2016):

1. First Stable Release: Marked the language as production-ready after years of development and refinement.
2. Core Features: Included null safety, extension functions, data classes, and higher-order functions.
3. Interoperability: Ensured 100% compatibility with Java, allowing developers to call Kotlin code from Java and vice versa
4. Google I/O Announcement: Google declared official support for Kotlin on Android, making it a first-class language for Android app development. Led to a significant surge in Kotlin adoption within the Android community.

Kotlin 1.1 (March 2017):

1. Coroutines (Experimental): Introduced coroutines for asynchronous programming, allowing developers to write non-blocking code more easily.
2. JavaScript Target: Enabled compilation of Kotlin code to JavaScript, facilitating cross-platform development.

Kotlin 1.2 (November 2017):

1. Multiplatform Projects (Experimental): Allowed sharing code between JVM and JavaScript platforms, paving the way for true cross-platform applications.
2. Improved Compilation: Enhanced compiler performance and incremental compilation support.

Kotlin 1.3 (October 2018):

1. Coroutines Become Stable: Solidified coroutines as a core feature, providing a powerful tool for asynchronous and concurrent programming.
2. Kotlin/Native: Enabled compilation to native binaries, expanding Kotlin's reach to platforms like iOS, Windows, Linux, and macOS without the need for a virtual machine.
3. Contracts: Introduced experimental support for contracts, allowing for more precise code analysis.

Kotlin 1.4 (August 2020):

1. Multiplatform Enhancements: Improved the multiplatform project support, making it more stable and easier to use.
2. Compiler Improvements: Focused on performance, resulting in faster compilation times and better IDE responsiveness.
3. Standard Library Updates: Added new functions and classes to the standard library, enhancing functionality.

Kotlin 1.5 (May 2021):

1. Language Features: Introduced JVM records, sealed interfaces, and inline classes.
2. Stability: Many experimental features were promoted to stable status.

Kotlin 1.6 (November 2021):

1. Standard Library Enhancements: Improved existing APIs and added new ones.
2. Performance: Continued focus on compiler and runtime performance optimizations.

Kotlin 1.7 (June 2022):

1. Context Receivers (Experimental): Added support for context-dependent declarations.
2. K2 Compiler (Alpha): Began work on a new frontend compiler aimed at performance improvements and better tooling.

Kotlin 1.8 (January 2023):

1. Incremental Updates: Brought further enhancements to the language and tooling.
2. Kotlin Multiplatform Mobile (KMM): Progressed towards stabilizing shared code between Android and iOS.

Kotlin 1.9 (July 2023):

1. K2 Compiler Advances: Continued development of the new compiler, improving compilation times and error diagnostics.
2. Language Features: Added new experimental features and made existing ones more stable

Kotlin 2.x (May 2024):

Kotlin 2.0 brings significant improvements and new features that make it a more powerful, expressive, and developer-friendly language. Here are some of the significant features introduced in Kotlin 2.0:

1. K2 Compiler (New Generation Compiler)
   1. Improved Performance: The new K2 compiler is designed to be faster and more efficient, significantly reducing compilation times.
   2. Improved Error Reporting: K2 enhances error diagnostics, providing more detailed and user-friendly error messages.
   3. Unified Backend: It unifies the backend of Kotlin/Native, Kotlin/JVM, and Kotlin/JS, allowing developers to work with different platforms more seamlessly.
   4. Modular and Extensible: The new architecture allows for easier integration of third-party tools, opening doors for more customization and extension.
2. Context Receivers
   1. This feature allows adding additional context to functions without explicitly passing it through parameters. It simplifies code that requires multiple receivers, such as in DSLs (Domain Specific Languages) and other multi-receiver scenarios.
      

      fun  withDatabaseContext(block: Database.() -> T): T { 
          return Database().block()
      }
      
      val result = withDatabaseContext {
          // Inside this lambda, `this` refers to a `Database` instance.
          query("SELECT * FROM users")
      }

3. Builder Inference Improvements: Kotlin 2.0 improves type inference in builder-style APIs, making code more concise and readable. This is particularly useful for libraries like coroutines and UI libraries that leverage builder patterns.
4. Explicit API Mode for Kotlin Libraries: The explicit API mode helps developers build public libraries with stricter controls. It enforces that every member of a public API must explicitly declare its visibility and type, making the API more predictable and well-documented.
5. Improved Multiplatform Support
   1. Multiplatform Compose: Kotlin 2.0 enhances Kotlin Multiplatform projects, especially for shared UI development. It supports libraries like Compose Multiplatform for building UIs that run on multiple platforms with the same codebase.
   2. Better Gradle Integration: The new version comes with improved Gradle tooling and support for managing multiple targets more easily.
6. New Sealed Interface Features: Kotlin 2.0 allows sealed interfaces, which, like sealed classes, restrict which classes can implement them. This improves safety in scenarios where specific hierarchies are needed.
   

   sealed interface Operation
   class Add(val value: Int) : Operation
   class Subtract(val value: Int) : Operation

7. Collection Literals and Destructuring in Loops: 

   1. Kotlin 2.0 introduces collection literals to make collection initialization more concise.

   2. Destructuring in loops is improved, allowing better handling of pairs and other data structures in iteration contexts.

8. Value Classes (Refined): Kotlin 2.0 continues to improve value classes (formerly inline classes), ensuring that they are more efficient and flexible. Value classes can be used in cases where a small, immutable data holder is needed without the overhead of full object allocation.
9. Unit Testing Enhancements: Kotlin 2.0 improves unit testing capabilities for Kotlin Multiplatform projects, providing better tools and frameworks for cross-platform test sharing and execution.
10. Enhanced Coroutines:
    1. Kotlin 2.0 further enhances Kotlin’s popular coroutine library with better integration, new debugging tools, and optimizations for more efficient asynchronous programming.
    2. Structured Concurrency Improvements: Kotlin 2.0 adds additional safeguards and features to make structured concurrency even more robust.
11. Function Interfaces: Kotlin 2.0 introduces Function Interfaces (aka SAM conversions), which allow developers to convert interfaces with a single abstract method into lambda expressions. This is especially useful when interoperating with Java libraries that heavily use functional interfaces.
12. Improved Null Safety Features: Kotlin 2.0 strengthens null-safety features, particularly in interoperability with Java. Improved type-checking mechanisms ensure that fewer null-pointer exceptions occur when interacting with non-null Kotlin code and nullable Java code.
13. Incremental Compilation Improvements: Kotlin 2.0 improves the incremental compilation process, reducing build times even for large projects and making the development experience smoother.
14. Backward Compatibility with Kotlin 1.x: Kotlin 2.0 is designed to be backward compatible with Kotlin 1.x, allowing gradual migration of projects without major breaking changes.

In both Java and Kotlin, the main function serves as the entry point of the application. However, the syntax and structure differ between the two languages.

In Java, the main method must be declared within a class and must be public, static, and void, accepting a String[] argument:

public class Main {
    public static void main(String[] args) {
        System.out.println("Hello, Java!");
    }
}

In Kotlin, the main function does not need to be part of a class and can be declared at the top level: 

fun main(args: Array) {
    println("Hello, Kotlin!")
}

Kotlin introduces a more concise and expressive way to declare variables, emphasizing immutability and type inference.

1. val (Immutable): Declares a read-only variable whose value cannot be changed once assigned.
2. var (Mutable): Declares a variable whose value can be changed.

For example:

val name = "Alice"   // Immutable variable
var age = 30         // Mutable variable

Attempting to reassign name will result in a compile-time error. age can be reassigned to a different value.

In Kotlin, you can define constants using val and const val. Both are used for read-only values, but they differ in when and how their values are initialized and how they can be used.

val:

1. Declares a read-only property or variable.
2. The value is assigned at runtime.
3. Can be used anywhere in the code.
4. Can hold any type, including objects.

const val:

1. Declares a compile-time constant.
2. The value is assigned at compile time.
3. Must be a top-level or member of an object or companion object.
4. Can only hold primitive types and String.

Example:

const val MAX_COUNT = 100
const val APP_NAME = "MyApplication"

object Constants {
    const val PI = 3.1415926535
    const val E = 2.7182818284
}


class MathUtils {
    companion object {
        const val GOLDEN_RATIO = 1.6180339887
    }
}

// Accessing the constant
val ratio = MathUtils.GOLDEN_RATIO

Kotlin can infer the type of a variable from the assigned value, so specifying the type explicitly is optional. Examples:

val number = 42               // Type inferred as Int
val pi = 3.1415               // Type inferred as Double
var message: String = "Hello" // Type explicitly specified

Kotlin does not have primitive types in the same way Java does. All types are objects. However, the compiler optimizes to use primitive types where possible for performance.

1. Numeric Types: Byte, Short, Int, Long, Float, Double
2. Other Types: Char, Boolean, String

Example: 

val count: Int = 10
// Or with type inference
val count = 10

One of the most significant features of Kotlin is its approach to null safety, which helps prevent the dreaded NullPointerException.

By default, variables in Kotlin cannot hold a null value. To allow a variable to hold null, you need to declare it as nullable by adding a

?

var nonNullable: String = "Hello"
// nonNullable = null // Compile-time error

var nullable: String? = "Hello"
nullable = null       // Allowed

To safely access properties or methods of a nullable variable, use the safe call operator

?.

// If nullable is not null, length will hold the length of the string.
// If nullable is null, length will be null.
val length = nullable?.length

The Elvis operator

?:

// If expression is not null, result will be the value of expression.
// If expression is null, result will be defaultValue.
val result = expression ?: defaultValue

The non-null assertion operator !! in Kotlin is used to explicitly assert that a nullable variable or expression is not null. If it is null, the operator will throw a

KotlinNullPointerException

// If nullableValue is not null, nonNullableValue will hold its value.
// If nullableValue is null, a KotlinNullPointerException is thrown.
val nonNullableValue = nullableValue!!

Use

as?

val obj: Any = "Kotlin"
val str: String? = obj as? String

In Java, all object references can be null, and there's no language-level enforcement to prevent NullPointerException. 

String message = null;
int length = message.length(); // Throws NullPointerException

To address issues related to null values, Java 8 introduced the

Optional

Example of Using Optional:

import java.util.Optional;

Optional optionalMessage = Optional.ofNullable(getMessage());

if (optionalMessage.isPresent()) {
    int length = optionalMessage.get().length();
} else {
    int length = 0;
}

 Or using a functional style:

int length = optionalMessage.map(String::length).orElse(0);

Limitations of Optional in Java:

1. Not a Language-Level Feature: Optional is a library class, not a language construct. It doesn't enforce null safety at the type system level.
2. Limited Usage: It's primarily intended for return types and not recommended for fields or method parameters, limiting its scope.
3. Verbosity: Using Optional can make the code more verbose compared to Kotlin's null handling.
4. Performance Overhead: Wrapping values in Optional can introduce performance overhead due to additional object creation.

Kotlin allows embedding variables and expressions within strings using string templates.

1. Variable Interpolation: Use

   $variableName

2. Expression Interpolation: Use

   ${expression}

Example:

val name = "Alice"
val greeting = "Hello, $name!"
println(greeting) // Output: Hello, Alice!

val age = 30
println("In 5 years, ${name} will be ${age + 5} years old.")
// Output: In 5 years, Alice will be 35 years old.

In Kotlin, operators can be classified into various categories, similar to Java. However, there are some key differences. Below is a comprehensive table of Kotlin operators and their usage. Where applicable, significant differences between Kotlin and Java are highlighted.

a = b

a += b

a -= b

a *= b

a /= b

a %= b

a + b

a - b

a * b

a / b

a % b

a == b

a != b

a > b

a < b

a >= b

a <= b

a && b

a || b

!a

Unlike Java, Kotlin does not have specific bitwise operators (&, |, etc.). Instead, it uses functions like and(), or(), xor(), inv() for bitwise operations.

a.and(b)

a.or(b)

a.xor(b)

a.inv()

x in array

x !in array

x is String

x as String

foo(*args)

a?.length

a ?: "default"

a!!

a++

a--

1..5

::foo

array[0]

myFunction()

Understanding these operators, especially those unique to Kotlin, like the spread operator and null-safe operators, will make coding more efficient and error-free compared to Java.

Kotlin reserves certain keywords for defining syntax elements like functions, classes, and more. These keywords cannot be used as identifiers (such as variable or function names) unless escaped with backticks. Below is a table listing all Kotlin keywords and their purposes.

These keywords are essential for understanding the Kotlin language syntax. By knowing their purpose, you can write cleaner and more efficient Kotlin code. Some keywords like

val

var

fun

Control flow constructs are essential in any programming language as they dictate the order in which statements are executed. Kotlin offers a rich set of control flow statements that are both expressive and concise. In this section, we'll explore how Kotlin handles conditional statements and loops, highlighting the differences and similarities with Java.

Kotlin provides powerful conditional statements, including

if

when

In Kotlin, if and else are expressions, meaning they return a value. This allows you to assign the result of an if expression directly to a variable, enhancing code conciseness.

val result = if (condition) {
    // Block of code
    valueIfTrue
} else {
    // Block of code
    valueIfFalse
}

Example: 

val max = if (a > b) a else b

Equivalent Java Code:

int max = (a > b) ? a : b;

Kotlin allows for multiple else if branches within an if expression.

val grade = if (score >= 90) {
    "A"
} else if (score >= 80) {
    "B"
} else if (score >= 70) {
    "C"
} else if (score >= 60) {
    "D"
} else {
    "F"
}

Equivalent Java Code:

String grade;
if (score >= 90) {
    grade = "A";
} else if (score >= 80) {
    grade = "B";
} else if (score >= 70) {
    grade = "C";
} else if (score >= 60) {
    grade = "D";
} else {
    grade = "F";
}

Kotlin's when expression is a powerful and flexible construct that can replace Java's switch statement. Starting from Java 14, Java introduced switch expressions, which bring some of the capabilities of Kotlin's when to Java.

Syntax of Kotlin's when Expression:

when (expression) {
    value1 -> result1
    value2, value3 -> result2
    in range -> result3
    !in range -> result4
    is Type -> result5
    else -> defaultResult
}

Example:

fun determineResponse(input: Any): String {
    return when (input) {
        // Matching a specific value
        1 -> "You entered one."
        
        // Matching multiple values
        2, 3 -> "You entered two or three."

        // Matching a value within a range
        in 4..10 -> "Your number is in the range of 4 to 10."

        // Matching a value outside of a range
        !in 11..20 -> "Your number is not in the range of 11 to 20."

        // Matching by type
        is String -> "You entered a string."

        // Default case
        else -> "I don't know what you entered."
    }
}

Java 14 introduced switch expressions, which can return a value and use the arrow syntax -> 

int day = 3;

String dayName = switch (day) {
    case 1 -> "Monday";
    case 2 -> "Tuesday";
    case 3 -> "Wednesday";
    case 4 -> "Thursday";
    case 5 -> "Friday";
    case 6, 7 -> "Weekend";
    default -> "Invalid day";
};

System.out.println(dayName); // Output: Wednesday

Kotlin's when can perform type checks using

is

// Kotlin Example:
fun describe(obj: Any): String =
    when (obj) {
        is Int -> "Integer"
        is String -> "String of length ${obj.length}"
        is Boolean -> "Boolean"
        else -> "Unknown"
    }

println(describe("Hello")) // Output: String of length 5

// Java Example with Pattern Matching (instanceof) (Java 16 and Later):
public String describe(Object obj) {
    if (obj instanceof Integer) {
        return "Integer";
    } else if (obj instanceof String s) {
        return "String of length " + s.length();
    } else if (obj instanceof Boolean) {
        return "Boolean";
    } else {
        return "Unknown";
    }
}

// Java Example with Pattern Matching in switch (Java 17 Preview):
public String describe(Object obj) {
    return switch (obj) {
        case Integer i -> "Integer";
        case String s -> "String of length " + s.length();
        case Boolean b -> "Boolean";
        default -> "Unknown";
    };
}

Kotlin offers loops that are similar to Java's but with enhanced features and more concise syntax.

Kotlin's for loop is designed to iterate over any iterable, including ranges, arrays, and collections.

Iterating Over Ranges with Kotlin:

// Iterating Over Ranges
for (i in 1..5) {
    print("$i ") // Output: 1 2 3 4 5
}

Java Equivalent Using for Loop: 

for (int i = 1; i <= 5; i++) {
    System.out.print(i + " "); // Output: 1 2 3 4 5
}

Java Equivalent Using Streams (Java 8 and Later) :

IntStream.rangeClosed(1, 5).forEach(i -> System.out.print(i + " "));
// Output: 1 2 3 4 5

Iterating Over Collections with Kotlin:

val fruits = listOf("Apple", "Banana", "Cherry")

for (fruit in fruits) {
    println(fruit)
}

Java enhanced for loop:

List fruits = List.of("Apple", "Banana", "Cherry");

for (String fruit : fruits) {
    System.out.println(fruit);
}

These loops function similarly in both Kotlin and Java. 

var count = 5
while (count > 0) {
    println(count)
    count--
}

Kotlin provides range expressions that simplify loop constructs.

// Exclusive Range:
for (i in 1 until 5) {
    print("$i ") // Output: 1 2 3 4
}

// Downward Range:
for (i in 5 downTo 1) {
    print("$i ") // Output: 5 4 3 2 1
}


// Stepped Range:
for (i in 1..10 step 2) {
    print("$i ") // Output: 1 3 5 7 9
}

Java can use IntStream with methods like range, rangeClosed, and custom steps. 

IntStream.iterate(1, i -> i + 2)
    .limit(5)
    .forEach(i -> System.out.print(i + " "));
// Output: 1 3 5 7 9

Kotlin allows labeled break and continue statements for controlling nested loops.

outer@ for (i in 1..5) {
    for (j in 1..5) {
        if (i * j > 10) break@outer
        println("i = $i, j = $j")
    }
}

Java Equivalent Using for labeled loops:

outer: // Label for the outer loop
for (int i = 1; i <= 5; i++) {
    for (int j = 1; j <= 5; j++) {
        if (i * j > 10) {
            break outer; // Break the outer loop using the label
        }
        System.out.println("i = " + i + ", j = " + j);
    }
}

Exception handling in Kotlin is similar to Java but with some key differences.

try {
    // Code that may throw an exception
} catch (e: ExceptionType) {
    // Handle exception
} finally {
    // Optional finally block
}

Example:

try {
    val result = numerator / denominator
} catch (e: ArithmeticException) {
    println("Cannot divide by zero")
} finally {
    println("Execution completed")
}

Kotlin:

1. All exceptions are unchecked.
2. No need to declare exceptions with throws.
3. Reduces boilerplate code.

Java:

1. Differentiates between checked and unchecked exceptions.
2. Checked exceptions must be declared or handled.
3. Can lead to verbose code with try-catch blocks.

Java 7 introduced the try-with-resources statement to manage resources automatically.

try (BufferedReader reader = new BufferedReader(new FileReader("file.txt"))) {
    String line;
    while ((line = reader.readLine()) != null) {
        System.out.println(line);
    }
} catch (IOException e) {
    e.printStackTrace();
}

Kotlin provides the

use

BufferedReader(FileReader("file.txt")).use { reader ->
    var line = reader.readLine()
    while (line != null) {
        println(line)
        line = reader.readLine()
    }
}

The use function ensures the resource is closed after use. 

Kotlin's coroutines provide advanced exception handling mechanisms. Exceptions in coroutines can be handled within the coroutine or propagated to the caller.

import kotlinx.coroutines.*

fun main() = runBlocking {
    val job = launch {
        try {
            // Coroutine code that may throw an exception
        } catch (e: Exception) {
            // Handle exception
        }
    }
    job.join()
}

Java's Approach to Asynchronous Exception Handling:  Java uses CompletableFuture and ExecutorService for asynchronous programming, with exception handling mechanisms.

CompletableFuture future = CompletableFuture.runAsync(() -> {
    try {
        // Asynchronous code
    } catch (Exception e) {
        // Handle exception
    }
});

future.join();

Functions are fundamental building blocks in Kotlin, and they come with a variety of features that enhance code readability, conciseness, and expressiveness. In this section, we'll explore how functions in Kotlin differ from those in Java.

In Kotlin, functions are declared using the

fun

fun functionName(parameter1: Type1, parameter2: Type2): ReturnType {
    // function body
    return result
}

// Example
fun add(a: Int, b: Int): Int {
    return a + b
}

For functions that return a single expression, Kotlin allows you to simplify the syntax using the equals sign =.

fun multiply(a: Int, b: Int): Int = a * b
// If the return type can be inferred, you can omit it:
fun multiply(a: Int, b: Int) = a * b

In Java, methods must be declared within a class, and the syntax is more verbose.

public class Calculator {
    public int add(int a, int b) {
        return a + b;
    }
}

Starting from Java 8, you can define static methods in interfaces and use lambda expressions, but you still need to define methods within a class or interface.

Java Example with Lambda (Java 8 and Later):

BiFunction<Integer, Integer, Integer> add = (a, b) -> a + b;
int sum = add.apply(5, 3);

Kotlin allows you to specify default values for function parameters. If an argument is not provided, the default value is used.

fun greet(name: String = "Guest") {
    println("Hello, $name!")
}

greet()            // Output: Hello, Guest!
greet("Alice")     // Output: Hello, Alice!

When calling a function, you can specify the names of the parameters, allowing you to pass arguments in any order and enhance code readability.

fun displayInfo(name: String, age: Int, country: String) {
    println("$name is $age years old from $country.")
}

displayInfo(age = 30, country = "USA", name = "Bob")

In Java, prior to version 15, there is no direct support for default or named arguments in methods. You typically overload methods to achieve similar functionality.

Java Example with Method Overloading:

public void greet() {
    greet("Guest");
}

public void greet(String name) {
    System.out.println("Hello, " + name + "!");
}

Named arguments are not supported in Java. However, starting from Java 15, the introduction of Records allows for more concise data carriers, but they don't provide named arguments for methods.

// Example Using Records in Java 15
public record Person(String name, int age) {}

public class Main {
    public static void main(String[] args) {
        // Creating an instance of Person
        Person person = new Person("Alice", 30);
        displayInfo(person);
    }

    public static void displayInfo(Person person) {
        System.out.println(person.name() + " is " + person.age() + " years old.");
    }
}

Extension functions in Kotlin allow you to add new functions to existing classes without inheriting from them or using design patterns like Decorator.

fun String.isPalindrome(): Boolean {
    return this == this.reversed()
}

val word = "level"
println(word.isPalindrome()) // Output: true

Java does not support extension functions directly. To achieve similar functionality, you would create utility classes with static methods.

Java Example:

public class StringUtils {
    public static boolean isPalindrome(String s) {
        return s.equals(new StringBuilder(s).reverse().toString());
    }
}

String word = "level";
System.out.println(StringUtils.isPalindrome(word)); // Output: true

With Java 8 and later, you can use default methods in interfaces to provide implementations, but this requires modifying the interface and doesn't allow adding methods to existing classes like String. 

Kotlin treats functions as first-class citizens, meaning you can store functions in variables, pass them as parameters, and return them from other functions.

A higher-order function is a function that takes functions as parameters or returns a function.

fun operate(a: Int, b: Int, operation: (Int, Int) -> Int): Int {
    return operation(a, b)
}

// ::add syntax stands for a function reference
val sum = operate(4, 5, ::add)
println(sum) // Output: 9

// Using lambda expression
// Trailing Lambda Syntax: if the last parameter of the function is a lambda, Kotlin allows 
// you to move the lambda outside of the parentheses for improved readability:
val product = operate(4, 5) { x, y -> x * y }
println(product) // Output: 20

Lambda expressions provide a concise way to represent functions.

val lambdaName: (InputType) -> ReturnType = { arguments -> body }

Example:

val square: (Int) -> Int = { number -> number * number }
println(square(6)) // Output: 36

Kotlin provides a concise and expressive syntax for passing lambda expressions as function parameters. One of the features that enhance code readability is the trailing lambda syntax. This allows you to pass a lambda expression after the function call parentheses, which can make your code more readable, especially when working with higher-order functions.

// Standard syntax
functionName(parameters..., { lambda_parameters -> lambda_body })

// Trailing lambda syntax
functionName(parameters...) { lambda_parameters -> lambda_body }

// If the lambda is the only parameter
functionName { lambda_parameters -> lambda_body }

Examples:

fun performOperation(x: Int, operation: (Int) -> Int): Int {
    return operation(x)
}

// Standard Syntax
val result = performOperation(5, { num -> num * num })
println(result) // Output: 25


// Trailing Lambda Syntax
val result = performOperation(5) { num ->
    num * num
}
println(result) // Output: 25

If a function takes only a lambda parameter, you can omit the parentheses entirely.

fun repeatAction(times: Int, action: () -> Unit) {
    for (i in 1..times) {
        action()
    }
}

repeatAction(3) {
    println("Hello, World!")
}
// Output:
// Hello, World!
// Hello, World!
// Hello, World!

Kotlin's standard library provides many functions that take lambdas as parameters. Trailing lambdas can make these calls more readable.

val numbers = listOf(1, 2, 3, 4, 5)

// Standard syntax
val evenNumbers = numbers.filter({ it % 2 == 0 })

// Trailing lambda syntax
val evenNumbers = numbers.filter { it % 2 == 0 }

println(evenNumbers) // Output: [2, 4]


val result = numbers
    .filter { it > 2 }
    .map { it * it }
    .also { println("Squared numbers: $it") }

In lambdas with a single parameter, you can omit the parameter declaration and use the implicit

it

val squares = numbers.map { it * it }
println(squares) // Output: [1, 4, 9, 16, 25]
 

Kotlin allows you to declare functions as

inline

inline fun performOperation(a: Int, b: Int, operation: (Int, Int) -> Int): Int {
    return operation(a, b)
}

Java 8 introduced lambda expressions and functional interfaces, enabling functional programming paradigms.

BiFunction<Integer, Integer, Integer> add = (a, b) -> a + b;
int sum = add.apply(4, 5);
System.out.println(sum); // Output: 9

Java doesn't support higher-order functions in the same way as Kotlin. You can pass functional interfaces as parameters, but the syntax is more verbose. 

public int operate(int a, int b, BiFunction<Integer, Integer, Integer> operation) {
    return operation.apply(a, b);
}

int product = operate(4, 5, (x, y) -> x * y);
System.out.println(product); // Output: 20

Kotlin supports tail recursive functions using the

tailrec

tailrec fun factorial(n: Int, accumulator: Int = 1): Int {
    return if (n <= 1) accumulator else factorial(n - 1, n * accumulator)
}

println(factorial(5)) // Output: 120

The tailrec modifier in Kotlin transforms a recursive function into an iterative one during compilation, optimizing it to prevent stack overflow issues that can occur with deep recursion. This is achieved by performing tail call optimization (TCO). Here's a breakdown of the exact effect of the tailrec keyword:

1. Tail Call Optimization (TCO): In Kotlin, a function call is considered a tail call if it's the last operation to be executed in a function. If the recursive call is in the tail position (i.e., it is the last thing the function does before returning), Kotlin replaces the recursive call with a loop during compilation, avoiding the need for additional stack frames for each recursive call.
2. Stack Frame Elimination: Normally, every function call adds a new stack frame, which can lead to stack overflow errors for deep recursion. The tailrec modifier removes the need for these additional frames by reusing the current function’s stack frame.
3. Iterative Approach: When you mark a function with tailrec, the compiler rewrites the recursive function into a loop under the hood. This makes the recursion as efficient as a traditional loop, avoiding the overhead of managing recursion in the stack.

Java does not have built-in support for tail call optimization. Recursive functions can lead to stack overflow errors if not carefully managed.

Kotlin allows you to define functions inside other functions, and these inner functions can access variables from the outer function.

fun greeting(): () -> Unit {
    val message = "Hello"
    fun sayHello() {
        println(message)
    }
    return ::sayHello
}

val greet = greeting()
greet() // Output: Hello

A function inside another function is called a Closure because it "close over" its surrounding environment, meaning it captures and remembers the state of variables from the outer function, even after that outer function has finished executing.

Java supports lambda expressions and anonymous inner classes but does not support defining named local functions inside methods.

Kotlin allows you to define lambda expressions that have a receiver object, enabling a DSL-like syntax.

// The lambda (builderAction) is an extension function on StringBuilder that doesn't return any value (Unit).
// Unit is a special type that is used to indicate the absence of a meaningful return value from a function. 
// It is similar to void in languages like Java or C, but it is treated as a real type in Kotlin, and it has
// only one value, which is Unit
fun buildString(builderAction: StringBuilder.() -> Unit): String {
    val sb = StringBuilder()
    sb.builderAction()
    return sb.toString()
}

val result = buildString {
    append("Hello, ")
    append("World!")
}

println(result) // Output: Hello, World!

The

{ ... }

Java does not support function literals with receivers. Achieving similar functionality would require more verbose code and design patterns.

Kotlin allows you to provide implementations for predefined operators for your own types by overloading them. 

// In Kotlin, a data class is a special type of class that is primarily used to hold data. 
// It automatically generates useful methods like equals(), hashCode(), toString(), and copy() for you, 
// based on the properties you define. This makes it especially handy when creating simple classes whose 
// main purpose is to store state (i.e., the values of its properties).
data class Point(val x: Int, val y: Int) {
    operator fun plus(other: Point) = Point(x + other.x, y + other.y)
}

val p1 = Point(2, 3)
val p2 = Point(4, 5)
val sum = p1 + p2
println(sum) // Output: Point(x=6, y=8)

Java does not support operator overloading, except for the + operator for string concatenation. You would need to define methods like add to achieve similar functionality. 

Although coroutines are covered in detail in a later section, it's worth mentioning that Kotlin functions can be declared with the

suspend

suspend fun fetchData(): String {
    // Simulate a long-running operation
    delay(1000)
    return "Data fetched"
}

suspend functions are special functions in Kotlin that can be paused and resumed at a later time without blocking the thread they are running on.

To use the fetchData() function, you'll need to call it within a coroutine scope, as suspend functions can only be called from within another suspend function or a coroutine.

import kotlinx.coroutines.*

fun main() {
    // Start a new coroutine:  This creates a coroutine scope that blocks the main thread 
    // until all coroutines inside it complete.
    runBlocking {
        println("Fetching data...")
        val result = fetchData() // Calls the suspend function
        println(result) // Prints: Data fetched
    }
}

Starting from Java 19, Virtual Threads (Project Loom) have been introduced to support lightweight concurrency. However, as of Java 21, coroutines as language constructs are not available. Asynchronous programming in Java is typically handled using CompletableFuture, reactive streams, or third-party libraries.

Java Example with CompletableFuture:

CompletableFuture future = CompletableFuture.supplyAsync(() -> {
    // Simulate long-running operation
    Thread.sleep(1000);
    return "Data fetched";
});

future.thenAccept(System.out::println);

Kotlin provides several special functions like

with

apply

run

let

The

with

data class User(val name: String, var age: Int, var city: String)

fun main() {
    val user = User("John", 25, "New York")
    with(user) {
        println(name)  // Access `name` directly
        age += 1
        println("Updated age: $age")
    }
}

Output:

John
Updated age: 26

In the example,

with

user

The

apply

val user = User("John", 25, "New York").apply {
    age = 26
    city = "San Francisco"
}
println(user)  // Output: User(name=John, age=26, city=San Francisco)

The

apply

The

let

?.

val name: String? = "John"
name?.let {
    println("Hello, $it")  // Output: Hello, John
}

If

name

let

The

run

let

val user = User("John", 25, "New York")
val result = user.run {
    "User's name is $name, age is $age"
}
println(result)  // Output: User's name is John, age is 25

In this example,

run

The

also

apply

val user = User("John", 25, "New York").also {
    println("User created: $it")
}
println(user)  // Output: User(name=John, age=25, city=New York)

also

The

takeIf

takeUnless

val user = User("John", 25, "New York")

val result = user.takeIf { it.age >= 18 }  // Returns user if age >= 18
println(result)  // Output: User(name=John, age=25, city=New York)

val resultNull = user.takeUnless { it.age < 18 }  // Returns user if age >= 18
println(resultNull)  // Output: User(name=John, age=25, city=New York)

Both

takeIf

takeUnless

These special functions in Kotlin provide a powerful way to write concise and expressive code. By using these functions effectively, you can avoid boilerplate code and make your programs easier to understand and maintain.

Object-Oriented Programming (OOP) is a paradigm centered around objects and classes, enabling developers to model real-world entities and relationships in code. Both Kotlin and Java are object-oriented languages, but Kotlin introduces several enhancements and syntactic sugar to make OOP more concise and expressive. This section explores how Kotlin handles OOP concepts compared to Java. 

In Kotlin, classes are declared using the

class

Example:

class Person {
    var name: String = ""
    var age: Int = 0

    fun greet() {
        println("Hello, my name is $name.")
    }
}

Equivalent Java code:

public class Person {
    private String name = "";
    private int age = 0;

    public void greet() {
        System.out.println("Hello, my name is " + name + ".");
    }
}

Kotlin introduces the concept of primary constructors, which are declared in the class header and can initialize properties directly.

// val name: String declares an immutable property.
// var age: Int declares a mutable property.
// Properties are initialized through the constructor.
class Person(val name: String, var age: Int) {
    fun greet() {
        println("Hello, my name is $name.")
    }
}

// Instantiate a person
val personInstance = Person(nameArgument, ageArgument)

Equivalent Java code: 

public class Person {
    private final String name;
    private int age;

    public Person(String name, int age) {
        this.name = name;
        this.age = age;
    }
    
    public void greet() {
        System.out.println("Hello, my name is " + name + ".");
    }
    
    // Getters and setters omitted for brevity
}

// Instantiate a person
Person personInstance = new Person(nameArgument, ageArgument);

In Kotlin, initializer blocks can be used to execute code during object creation.

class Person(val name: String, var age: Int) {
    init {
        println("Person initialized with name = $name and age = $age")
    }
}

Kotlin allows secondary constructors for additional initialization logic. 

class Person {
    var name: String
    var age: Int

    constructor(name: String) {
        this.name = name
        this.age = 0
    }

    constructor(name: String, age: Int) {
        this.name = name
        this.age = age
    }
}

Please note that Secondary Constructors are less common due to the flexibility of default parameters.

In Kotlin, properties are a central concept that combines a field (to hold data) and optional accessors (getters and setters) into a single, concise syntax. This approach simplifies code and reduces boilerplate compared to Java, where you typically need to declare private fields and provide public getter and setter methods separately.

//  declares a mutable property name of type String with an initial value of "Unknown".
var name: String = "Unknown"

For a mutable property declared with

var

1. A getter method to retrieve the property's value.
2. A setter method to set or modify the property's value.

For an immutable property declared with

val

You can customize the getter and setter if you need additional logic when accessing or modifying the property. Syntax for Custom Accessors:

var propertyName: Type = initialValue
    get() {
        // Custom getter logic
    }
    set(value) {
        // Custom setter logic
    }

Within the getter and setter,

field

var name: String = "Unknown"
    get() = field
    set(value) {
        field = value.capitalize()
    }

In Java, we always need to explicitly implement getter and setter:

private String name = "Unknown";

public String getName() {
    return name;
}

public void setName(String name) {
    this.name = capitalize(name);
}

private String capitalize(String value) {
    // Capitalize logic
}

In Kotlin, classes are final by default. To allow a class to be subclassed, you must mark it with the

open

open class Animal {
    open fun sound() {
        println("Some sound")
    }
}

class Dog : Animal() {
    override fun sound() {
        println("Bark")
    }
}

In Java, classes and methods are open for extension by default unless marked as final:

public class Animal {
    public void sound() {
        System.out.println("Some sound");
    }
}

public class Dog extends Animal {
    @Override
    public void sound() {
        System.out.println("Bark");
    }
}

Kotlin interfaces can contain both abstract methods and method implementations.

interface Movable {
    fun move()
    fun stop() {
        println("Stopped moving")
    }
}

class Vehicle : Movable {
    override fun move() {
        println("Vehicle is moving")
    }
}

From Java 8,  interfaces can provide default implementations:

public interface Movable {
    void move();
    
    default void stop() {
        System.out.println("Stopped moving");
    }
}

public class Vehicle implements Movable {
    @Override
    public void move() {
        System.out.println("Vehicle is moving");
    }
}

Kotlin allows a class to implement multiple interfaces, and if there are conflicts, you must override the conflicting methods.

interface A {
    fun show() {
        println("A")
    }
}

interface B {
    fun show() {
        println("B")
    }
}

class C : A, B {
    override fun show() {
        super.show()
        super.show()
    }
}

val c = C()
c.show()
// Output:
// A
// B

In Java you need to do something similar to solve the conflict:

public interface A {
    default void show() {
        System.out.println("A");
    }
}

public interface B {
    default void show() {
        System.out.println("B");
    }
}

public class C implements A, B {
    @Override
    public void show() {
        A.super.show();
        B.super.show();
    }
}

Data classes in Kotlin are designed to hold data. The compiler automatically generates

equals()

hashCode()

toString()

copy()

data class User(val name: String, val age: Int)

val user1 = User("Alice", 30)
println(user1) // Output: User(name=Alice, age=30)

val user2 = user1.copy(age = 31)
println(user2) // Output: User(name=Alice, age=31)

Java Equivalent Prior to Records:

public class User {
    private final String name;
    private final int age;
    
    public User(String name, int age) {
        this.name = name;
        this.age = age;
    }
    
    // Getters, equals(), hashCode(), and toString() methods
    // Need to be manually implemented or generated by the IDE
}

Java Equivalent with Records ( Java 16+ ):

public record User(String name, int age) {}

User user1 = new User("Alice", 30);
System.out.println(user1); // Output: User[name=Alice, age=30]

User user2 = new User(user1.name(), 31);
System.out.println(user2); // Output: User[name=Alice, age=31]

Sealed classes and interfaces restrict the hierarchy to a finite set of subclasses, known at compile time.

sealed class Result

// Declares a data class Success that inherits from Result
data class Success(val data: String) : Result()
data class Error(val exception: Exception) : Result()
// Declares a singleton object Loading that inherits from Result.
object Loading : Result()

Java 17 introduced sealed classes and interfaces. 

public sealed class Result permits Success, Error, Loading {}

public final class Success extends Result {
    private final String data;
    // Constructor, getters, etc.
}

public final class Error extends Result {
    private final Exception exception;
    // Constructor, getters, etc.
}

public final class Loading extends Result {}

Kotlin offers several visibility modifiers:

1. public (default): Visible everywhere.
2. internal: Visible within the same module.
3. protected: Visible to subclasses.
4. private: Visible within the containing declaration.

Example:

class Example {
    private val x = 1
    internal val y = 2
    protected val z = 3
    val w = 4 // Public by default
}

Java Visibility Modifiers: 

1. public: Visible everywhere.
2. protected: Visible within the package and subclasses.
3. Package-private (default): Visible within the package.
4. private: Visible within the class.

In Kotlin, a nested class is static by default. It does not hold a reference to the outer class.

class Outer {
    class Nested {
        fun hello() = "Hello from Nested"
    }
}

val message = Outer.Nested().hello()
println(message) // Output: Hello from Nested

Java static netsted class:

public class Outer {
    public static class Nested {
        public String hello() {
            return "Hello from Nested";
        }
    }
}

String message = new Outer.Nested().hello();
System.out.println(message); // Output: Hello from Nested

An inner class in Kotlin holds a reference to the outer class and can access its members.

class Outer(val name: String) {
    inner class Inner {
        fun greet() = "Hello from $name's Inner"
    }
}

val outer = Outer("Kotlin")
val message = outer.Inner().greet()
println(message) // Output: Hello from Kotlin's Inner

Java Inner Class:

public class Outer {
    private String name;

    public Outer(String name) {
        this.name = name;
    }

    public class Inner {
        public String greet() {
            return "Hello from " + name + "'s Inner";
        }
    }
}

Outer outer = new Outer("Java");
String message = outer.new Inner().greet();
System.out.println(message); // Output: Hello from Java's Inner

Data classes can also be nested or inner class:

// Nested
data class Person(val name: String, val age: Int) {
    data class Address(val street: String, val city: String)
}

fun main() {
    val address = Person.Address("Main St", "Springfield")
    val person = Person("John Doe", 30)

    println(person)  // Output: Person(name=John Doe, age=30)
    println(address) // Output: Address(street=Main St, city=Springfield)
}


// Inner
data class Person(val name: String, val age: Int) {
    inner data class Address(val street: String, val city: String) {
        fun getFullAddress(): String {
            return "$name lives at $street, $city"  // Accessing outer class (Person) properties
        }
    }
}

fun main() {
    val person = Person("John Doe", 30)
    val address = person.Address("Main St", "Springfield")

    println(address.getFullAddress())  // Output: John Doe lives at Main St, Springfield
}

Kotlin provides a concise way to create singleton objects using

object

object Singleton {
    fun greet() = "Hello from Singleton"
}

println(Singleton.greet()) // Output: Hello from Singleton

 In Java, you typically use a class with a private constructor and a static instance.

public class Singleton {
    private static final Singleton INSTANCE = new Singleton();
    
    private Singleton() {}

    public static Singleton getInstance() {
        return INSTANCE;
    }

    public String greet() {
        return "Hello from Singleton";
    }
}

System.out.println(Singleton.getInstance().greet()); // Output: Hello from Singleton

In Kotlin, companion objects can hold static members and factory methods.

class MyClass {
    companion object {
        const val CONSTANT = 100
        fun create(): MyClass = MyClass()
    }
}

println(MyClass.CONSTANT) // Output: 100
val instance = MyClass.create()

Java uses static members and methods.

public class MyClass {
    public static final int CONSTANT = 100;

    public static MyClass create() {
        return new MyClass();
    }
}

System.out.println(MyClass.CONSTANT); // Output: 100
MyClass instance = MyClass.create();

Kotlin supports delegation of interface implementation to another object.

interface Printer {
    fun print()
}

class ConsolePrinter : Printer {
    override fun print() {
        println("Printing to console")
    }
}

class PrinterManager(printer: Printer) : Printer by printer

val manager = PrinterManager(ConsolePrinter())
manager.print() // Output: Printing to console

The syntax 

Printer by printer

Java does not have built-in support for class delegation. You would need to implement the methods and delegate calls manually.

public interface Printer {
    void print();
}

public class ConsolePrinter implements Printer {
    @Override
    public void print() {
        System.out.println("Printing to console");
    }
}

public class PrinterManager implements Printer {
    private final Printer printer;

    public PrinterManager(Printer printer) {
        this.printer = printer;
    }

    @Override
    public void print() {
        printer.print();
    }
}

PrinterManager manager = new PrinterManager(new ConsolePrinter());
manager.print(); // Output: Printing to console

Kotlin also allows delegation of property getters and setters.

import kotlin.properties.Delegates

class User {
    // A mutable property (var) named name of type String
    // Use Kotlin's Delegates.observable delegate to manage the behavior of the name property. 
    //   This function allows you to observe changes to the property and take some action 
    //   whenever the property's value is changed.
    // The initial value of the name property is set to ""
    // The last parameter of Delegates.observable is a
    var name: String by Delegates.observable("") { prop, old, new ->
        println("Property '${prop.name}' changed from '$old' to '$new'")
    }
}

val user = User()
user.name = "Alice" // Output: Property 'name' changed from '' to 'Alice'

Abstract classes in Kotlin are declared using the abstract keyword and can contain abstract members that must be implemented by subclasses.

abstract class Vehicle {
    abstract fun drive()
    fun stop() {
        println("Vehicle stopped")
    }
}

class Car : Vehicle() {
    override fun drive() {
        println("Car is driving")
    }
}

val car = Car()
car.drive() // Output: Car is driving
car.stop()  // Output: Vehicle stopped

Equivalent Java code:

public abstract class Vehicle {
    public abstract void drive();
    
    public void stop() {
        System.out.println("Vehicle stopped");
    }
}

public class Car extends Vehicle {
    @Override
    public void drive() {
        System.out.println("Car is driving");
    }
}

Car car = new Car();
car.drive(); // Output: Car is driving
car.stop();  // Output: Vehicle stopped

Enum classes represent a fixed set of constants.

enum class Direction {
    NORTH, SOUTH, EAST, WEST
}

val dir = Direction.NORTH
println(dir) // Output: NORTH

Java Enum Class:

public enum Direction {
    NORTH, SOUTH, EAST, WEST;
}

Direction dir = Direction.NORTH;
System.out.println(dir); // Output: NORTH

Kotlin introduces inline classes (now known as value classes) to create type-safe wrappers without runtime overhead.

Kotlin Value Class (Kotlin 1.5 and Later):

@JvmInline
value class Email(val address: String)

fun sendEmail(email: Email) {
    // ...
}

val email = Email("test@example.com")
sendEmail(email)

In ths example above, Email is a wrapper around String but without additional allocation.

Kotlin distinguishes between structural equality (==) and referential equality (===).

1. a == b

   checks if the values are equal (calls equals()).

2. a === b

   checks if the references are the same.

Java Equivalent:

1. a.equals(b)

   checks value equality.

2. a == b

   checks reference equality.

Coroutines are a powerful feature in Kotlin that facilitate asynchronous and non-blocking programming. They are lightweight threads that allow you to write asynchronous code in a sequential and readable manner.

In traditional asynchronous programming, especially in Java, handling asynchronous tasks often leads to:

1. Callback Hell: Nested callbacks that make code hard to read and maintain.
2. Complex Thread Management: Manual handling of threads, synchronization, and locking mechanisms.

Kotlin coroutines simplify asynchronous programming by:

1. Suspending Functions: Functions that can suspend execution without blocking the thread.
2. Structured Concurrency: Managing coroutines in a structured way to avoid leaks and ensure proper cancellation.

1. Lightweight: Coroutines are much lighter than threads. You can run thousands of coroutines without significant overhead.
2. Non-blocking: Suspending a coroutine doesn't block the underlying thread, allowing other coroutines to run.
3. Simplified Syntax: Coroutines allow writing asynchronous code sequentially, improving readability.

Coroutine builders are functions that help you create and start coroutines.

Starts a new coroutine without blocking the current thread. It returns a Job that can be used to manage the coroutine.

import kotlinx.coroutines.*

fun main() = runBlocking {
    launch {
        delay(1000L)
        println("World!")
    }
    println("Hello,")
}
// Output:
// Hello,
// World!

Starts a new coroutine and returns a

Deferred

Future

import kotlinx.coroutines.*

fun main() = runBlocking {
    val deferred = async {
        delay(1000L)
        "Result"
    }
    println("Waiting for result...")
    val result = deferred.await()
    println("Result: $result")
}
// Output:
// Waiting for result...
// Result: Result

Bridges the gap between regular blocking code and suspending code. It blocks the current thread until its coroutine completes.

import kotlinx.coroutines.*

fun main() = runBlocking {
    println("Start")
    delay(1000L)
    println("End")
}
// Output:
// Start
// End

Functions marked with the suspend keyword that can suspend execution without blocking the thread. This kind of functions can only be called from within a coroutine or another suspending function.

suspend fun fetchData(): String {
    delay(1000L) // Simulate long-running task
    return "Data"
}

fun main() = runBlocking {
    val data = fetchData()
    println("Fetched: $data")
}

Coroutine scope defines the lifecycle of coroutines and provides context for them:

1. GlobalScope: Lives for the entire lifetime of the application:
   1. Coroutines launched in this scope live for the entire lifetime of the application.
   2. These coroutines are not bound to any specific lifecycle (like an activity or a function) and keep running unless explicitly canceled or when the application terminates.
   3. It should be avoided in most cases because it doesn't allow proper lifecycle management, which could lead to memory leaks or unintended behavior.
2. CoroutineScope: Custom scope for structured concurrency:
   1. It is used to create structured concurrency. This means coroutines launched within this scope are bound to its lifecycle, and if the scope is canceled, all the coroutines within it are also canceled.
   2. CoroutineScope can be manually created, or it can be inherited from a parent scope like in runBlocking.
   3. It is preferred for organizing coroutines so they can be properly canceled, ensuring that resources are not leaked.

Example of GlobalScope:

GlobalScope.launch {
    println("Running in GlobalScope")
}

 In this example, the coroutine runs independently of any lifecycle and will only stop when canceled or the application terminates.

Example of CoroutineScope inherited from a parent scope:

fun main() = runBlocking {
    launch { // Inherits parent scope
        println("Coroutine in runBlocking scope")
    }
}

runBlocking

Example of a custom scope: 

class MyClass {
    private val scope = CoroutineScope(Dispatchers.Default)

    fun startTask() {
        scope.launch {
            println("Running in a custom CoroutineScope")
        }
    }

    fun stopTask() {
        scope.cancel()  // Cancels all coroutines in this scope
    }
}

A coroutine context contains information like the job, dispatcher, and exception handler.

A dispatcher defines which thread or thread pool a coroutine will be executed on. Dispatchers help distribute and manage tasks efficiently based on their requirements, such as computational intensity, input/output operations, or user interface tasks:

1. Dispatchers.Default:
   1. Used for CPU-intensive tasks (e.g., complex calculations, sorting, etc.).
   2. It uses a shared pool of background threads optimized for CPU-bound operations.
2. Dispatchers.IO:
   1. Designed for I/O operations like reading from or writing to files, network requests, or database interactions.
   2. It uses a pool of threads optimized for blocking I/O operations to prevent CPU starvation.
3. Dispatchers.Main:
   1. Used for UI-related work, typically on the main thread of an application (for example, in Android development).
   2. It ensures that tasks interacting with the user interface are executed on the main thread to avoid UI lag or inconsistencies.

An example of using a dispatcher:

fun main() = runBlocking {
    launch(Dispatchers.IO) {
        val data = fetchData()
        println("Data: $data")
    }
}

Structured concurrency is a design principle in Kotlin Coroutines that ensures that coroutines are executed within a structured scope, with clear parent-child relationships. This design makes it easier to manage their lifecycle, handle exceptions, and ensure that resources are properly released.

In simple terms, structured concurrency ensures that coroutines have a well-defined lifecycle. The parent coroutine will not complete until all of its child coroutines have finished, and if the parent coroutine is canceled, all of its child coroutines will also be automatically canceled. This structure helps avoid common pitfalls such as leaking coroutines or leaving background tasks running indefinitely.

1. Automatic Cancellation: When a parent coroutine is canceled, all its child coroutines are canceled automatically. This prevents coroutines from running longer than necessary and ensures that resources (like memory or network connections) are freed up.
2. Avoiding Leaked Coroutines: Coroutines are tied to a scope, and the lifecycle of that scope dictates the lifetime of the coroutines. This avoids situations where coroutines continue running even after their associated tasks or parent coroutine is no longer needed, preventing resource leaks.
3. Lifecycle Management: The parent coroutine waits for its child coroutines to finish. This ensures a predictable and manageable lifecycle for the coroutines, avoiding orphaned coroutines that can be hard to trace and manage.

Example:

suspend fun fetchData(): String = coroutineScope {
    // Launch two async coroutines to fetch data concurrently
    val data1 = async { /* Simulate fetching data 1 */ "Data1" }
    val data2 = async { /* Simulate fetching data 2 */ "Data2" }

    // Wait for both data to be fetched and concatenate the results
    data1.await() + data2.await()
}

Explanation:

1. coroutineScope: The coroutineScope function creates a scope that ensures all the coroutines launched within it (like async or launch) are completed before it returns. This ensures that fetchData() will only return when both data1 and data2 have finished fetching their data.
2. async: The async function launches a new coroutine concurrently to execute a block of code. It's similar to launch, but it returns a Deferred result, which you can await to get the result.
3. await: This suspends the parent coroutine until the result of the child coroutine (created by async) is available. In this case, the result is the fetched data.
4. Structured Concurrency in Action:
   1. Both data1 and data2 are launched concurrently.
   2. The coroutineScope ensures that the parent coroutine waits for both data1.await() and data2.await() to finish before proceeding.
   3. If an exception occurs in one of the async blocks, or if the parent coroutine is canceled, both data1 and data2 coroutines will be automatically canceled.

1. Parent-Child Relationship: Every coroutine has a parent-child relationship when launched within a scope. The coroutineScope establishes this relationship.
2. Exception Propagation: If a child coroutine fails with an exception, that exception is propagated to its parent, and the entire coroutine scope is canceled unless handled explicitly. This ensures that errors are not silently ignored.
3. Job Hierarchy: Coroutines form a hierarchy of jobs, with a parent job being responsible for managing the completion of its children. Cancellation of the parent job cascades down to its children.

Both Channels and Flows provide ways to deal with asynchronous streams of data in Kotlin coroutines, but they have different use cases, behavior, and underlying mechanisms. Here's a breakdown of each, followed by the key differences.

1. Channels are similar to queues and provide a way for coroutines to send and receive data.
2. Channels allow bi-directional communication between producer and consumer coroutines, where one coroutine can send data and another coroutine can receive it.
3. Channels can be hot, meaning the data is produced whether or not the consumer is actively receiving it.
4. When a channel is closed (using channel.close()), it signals that no further values will be sent, but the consumer can still receive the remaining values until the channel is empty.

Example:

import kotlinx.coroutines.*
import kotlinx.coroutines.channels.Channel

fun main() = runBlocking {
    val channel = Channel()

    // Launching a coroutine to send data to the channel
    launch {
        for (x in 1..5) channel.send(x * x)  // Sending values (1^2, 2^2, ..., 5^2)
        channel.close()  // Close the channel to indicate no more data
    }

    // Receiving data from the channel
    for (y in channel) {
        println(y)  // Prints 1, 4, 9, 16, 25
    }
}

1. A Flow is a cold asynchronous data stream that emits values sequentially.
2. Flows are unidirectional and typically represent a one-way stream of data from producer to consumer.
3. Flows are cold, meaning the data is only produced when a consumer starts collecting the flow. If no one is collecting the flow, the producer is inactive.
4. Flows are much more declarative and follow a pattern similar to reactive streams. They emit data using the emit() function and are consumed using collect().
5. Flow APIs handle backpressure and cancellation transparently.

Example:

import kotlinx.coroutines.*
import kotlinx.coroutines.flow.*

fun simpleFlow(): Flow = flow {
    for (i in 1..3) {
        delay(100)  // Simulate asynchronous work
        emit(i)  // Emit values (1, 2, 3)
    }
}

fun main() = runBlocking {
    simpleFlow().collect { value ->
        println(value)  // Prints 1, 2, 3
    }
}

1. Threads: Heavyweight, managed by the OS.
2. Synchronization: Requires explicit handling of synchronization, locks, and potential deadlocks.
3. Asynchronous APIs: Use of Future, Callable, ExecutorService, and CompletableFuture. 

Example:

public class ThreadExample {
    public static void main(String[] args) {
        Thread thread = new Thread(() -> {
            try {
                Thread.sleep(1000);
                System.out.println("World!");
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        });
        thread.start();
        System.out.println("Hello,");
    }
}
// Output may vary due to thread scheduling:
// Hello,
// World!

CompletableFuture is for building asynchronous computation stages.

import java.util.concurrent.*;

public class CompletableFutureExample {
    public static void main(String[] args) throws Exception {
        CompletableFuture future = CompletableFuture.supplyAsync(() -> {
            try {
                Thread.sleep(1000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            return "Result";
        });
        System.out.println("Waiting for result...");
        String result = future.get();
        System.out.println("Result: " + result);
    }
}
// Output:
// Waiting for result...
// Result: Result

Java Virtual Threads are a feature introduced under Project Loom, available as a preview in Java 19 and beyond. Virtual Threads aim to provide lightweight, high-throughput threading by decoupling the notion of a Java thread from an operating system thread. Key features include:

1. Lightweight Threads: Virtual Threads are managed by the JVM rather than the OS, allowing for millions of threads.
2. Familiar APIs: Use the same java.lang.Thread API, making it easier for developers to adopt.
3. Better Resource Utilization: Improves scalability and performance for applications with high concurrency needs.
4. Compatibility: Works with existing Java code and libraries.

Example:

public class VirtualThreadExample {
    public static void main(String[] args) throws Exception {
        Thread.startVirtualThread(() -> {
            try {
                Thread.sleep(1000);
                System.out.println("World!");
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        });
        System.out.println("Hello,");
        Thread.sleep(1500); // Ensure the virtual thread has time to execute
    }
}

Generics are a powerful feature in Kotlin that allow you to write flexible and reusable code. By using generics, you can define classes, methods, and interfaces that work with any type while preserving type safety. Generics enable you to create algorithms or data structures that can handle multiple types without having to write the same logic multiple times for different types.

A generic class or function in Kotlin can accept a type parameter, which allows the user to specify the actual type when using that class or function. The type parameter is usually denoted by a single capital letter, commonly T, but you can use any name you like.

// Defining a generic class Box with a type parameter T
class Box(val value: T)

fun main() {
    // Creating a Box that holds an Int
    val intBox: Box = Box(42)
    println(intBox.value)  // Output: 42

    // Creating a Box that holds a String
    val stringBox: Box = Box("Hello")
    println(stringBox.value)  // Output: Hello
}

Just like classes, functions in Kotlin can also be made generic by introducing type parameters in the function definition. Here's how you can define and use a generic function: 

// Defining a generic function that works with any type
fun  printBoxContent(box: Box) {
    println("Box contains: ${box.value}")
}

fun main() {
    val intBox = Box(42)
    val stringBox = Box("Hello")
    
    // Calling the generic function with different types
    printBoxContent(intBox)    // Output: Box contains: 42
    printBoxContent(stringBox)  // Output: Box contains: Hello
}

Sometimes, you might want to restrict the types that can be used as generic arguments. Kotlin allows you to apply type constraints to limit the types that can be passed to a generic type parameter.

For example, you can limit a generic type to only accept subtypes of a particular class or implement a specific interface. This is done using the where keyword or inline directly after the type parameter with the

:

// Defining a generic class with a type constraint
class Container(val value: T)

fun main() {
    val intContainer = Container(42)   // Allowed because Int is a subtype of Number
    val doubleContainer = Container(3.14)  // Allowed because Double is a subtype of Number

    // val stringContainer = Container("Hello")  // Error: String is not a subtype of Number
}

Like Java, Kotlin uses type erasure for generics, meaning that the generic type information is only available at compile time and is erased at runtime. This means you cannot directly check the type of a generic parameter at runtime.

For instance, trying to check the type of a generic parameter like this will not work:

fun  checkType(value: T) {
    if (value is List) {
        // Error: Cannot check for List at runtime due to type erasure
    }
}

You can also define classes or functions with multiple generic type parameters, allowing even more flexibility.

// Defining a class with two generic type parameters
class PairBox<T1, T2>(val first: T1, val second: T2)

fun main() {
    val pair = PairBox(1, "One")
    println("First: ${pair.first}, Second: ${pair.second}")  // Output: First: 1, Second: One
}

In Kotlin, generics are usually erased at runtime, which means that type information is not available during runtime. However, in some cases, you may need to retain the generic type information for certain operations, such as casting or checking the type of an object. This is where the

reified

By using the

reified

inline

Normally, without

reified

reified

inline fun isTypeOf(value: Any): Boolean {
    return value is T
}

fun main() {
    val result = isTypeOf("Hello")
    println(result) // Output: true
}

In this example, the

isTypeOf

inline

reified

T

T

One practical use of

reified

reified

Without Reified:

fun  getClassName(clazz: Class): String {
    return clazz.simpleName
}

fun main() {
    val className = getClassName(String::class.java)
    println(className) // Output: String
}

With Reified:

inline fun  getClassName(): String {
    return T::class.java.simpleName
}

fun main() {
    val className = getClassName()
    println(className) // Output: String
}

As you can see, by using

reified

It’s important to note that the

reified

inline

Here’s an attempt to use

reified

fun  getClassName(): String { // Error: Reified type parameter T is not allowed in non-inline functions
    return T::class.java.simpleName
}

To summarize,

reified

In the context of programming languages that support generics (such as Kotlin, Java, Scala, etc.), variance  is a concept that describes how subtyping between more complex types (like generics or parameterized types) relates to subtyping between their components (like their type parameters).

Specifically, variance determines whether one generic type can be considered a subtype or supertype of another generic type based on their type parameters. For example, if you have two types

A

B

Box

Box

The

out

Box

Box

// Define a covariant Box class
class Box(val value: T)

fun main() {
    val intBox: Box = Box(42)
    val anyBox: Box = intBox  // This is allowed due to covariance (Int is a subtype of Any)

    // You can safely read from anyBox
    println(anyBox.value)  // Output: 42

    // However, you cannot modify anyBox because it's covariant
    // anyBox.value = "Hello"  // Error: Val cannot be reassigned
}

Here, Box is covariant in

T

Int

Any

Box

Box

Any

Box

Think of covariant types as producers. For example, a

Box

String

Any

Box

Box

Any

Contravariance is the opposite of covariance. If type

A

B

Box

Box

// Define a contravariant Box class
class Box {
    fun setValue(value: T) {
        println("Setting value: $value")
    }
}

fun main() {
    val anyBox: Box = Box()
    val stringBox: Box = anyBox  // This is allowed due to contravariance (String is a subtype of Any)

    // You can safely write a String to anyBox, because it's contravariant
    stringBox.setValue("Hello, World!")  // Output: Setting value: Hello, World!

    // However, you cannot read from stringBox because it's contravariant
    // val value: String = stringBox.value  // Error: Cannot read from a contravariant type
}

In this example,

Box

String

Any

Box

Box

String

Box

Think of contravariant types as consumers. A

Box

String

Box

Invariance means that there is no relationship between the subtyping of the type parameters and the subtyping of the generic types themselves. In other words, if type

A

B

Box

Box

// Define an invariant Box class
class Box(val value: T)

fun main() {
    val intBox: Box = Box(42)
    // val anyBox: Box = intBox  // Error: Type mismatch

    // You can both read and write values, but they must be of the exact type
    val anotherIntBox: Box = Box(100)
    println(anotherIntBox.value)  // Output: 100

    // If we had a setter (not shown here), it would only accept Int
    // anotherIntBox.setValue("Hello")  // Error: Type mismatch
}

Here,

Box

Box

Box

Int

Any

Invariant types are both producers and consumers of their exact type. You can both read and write values, but they must always be of the specific type

T

Collections are fundamental data structures that allow developers to store and manipulate groups of objects. Kotlin provides a rich set of collection APIs and functional operations that enhance productivity and code readability. In this section, we'll explore Kotlin's collections, compare them with Java's collection framework, and delve into functional operations that simplify common programming tasks.

 Kotlin collections are divided into two categories:

1. Immutable Collections: Read-only collections that cannot be modified after creation.
2. Mutable Collections: Collections that can be modified, allowing addition, removal, and updating of elements.

Immutable collections are preferred in functional programming paradigms as they prevent accidental modification and promote thread safety. 

Common Immutable Collection Types:

1. List: Ordered collection of elements.
2. Set: Unordered collection of unique elements.
3. Map: Collection of key-value pairs.

Example:

val numbers: List = listOf(1, 2, 3)

Mutable collections can be modified after creation.

Common Mutable Collection Types:

1. MutableList
2. MutableSet
3. MutableMap

Example:

val mutableNumbers: MutableList = mutableListOf(1, 2, 3)
mutableNumbers.add(4)

val fruits = listOf("Apple", "Banana", "Cherry")

val mutableFruits = mutableListOf("Apple", "Banana")
mutableFruits.add("Cherry")


val numbers = setOf(1, 2, 3, 2)
println(numbers) // Output: [1, 2, 3]

val mutableNumbers = mutableSetOf(1, 2, 3)
mutableNumbers.add(4)


val countryCodes = mapOf("US" to "United States", "CA" to "Canada")

val mutableCountryCodes = mutableMapOf("US" to "United States")
mutableCountryCodes["CA"] = "Canada"

Kotlin provides a plethora of functional operations that make working with collections more expressive and concise. These operations are inspired by functional programming paradigms.

1. map: Transforms each element.
   

   val numbers = listOf(1, 2, 3)
   val squares = numbers.map { it * it }
   println(squares) // Output: [1, 4, 9] 

2. filter: Filters elements based on a predicate.
   

   val numbers = listOf(1, 2, 3, 4, 5)
   val evenNumbers = numbers.filter { it % 2 == 0 }
   println(evenNumbers) // Output: [2, 4]

3. reduce: Reduces the collection to a single value.
   

   val numbers = listOf(1, 2, 3, 4)
   val sum = numbers.reduce { acc, num -> acc + num }
   println(sum) // Output: 10 

4. fold: Similar to reduce but with an initial value.
   

   val numbers = listOf(1, 2, 3, 4)
   val product = numbers.fold(1) { acc, num -> acc * num }
   println(product) // Output: 24

5. forEach: Performs an action on each element.
6. groupBy: Groups elements by a key.
   

   data class Person(val name: String, val city: String)
   
   val people = listOf(
       Person("Alice", "New York"),
       Person("Bob", "Paris"),
       Person("Charlie", "New York")
   )
   
   val peopleByCity = people.groupBy { it.city }
   println(peopleByCity)
   // Output:
   // {
   //   New York=[Person(name=Alice, city=New York), Person(name=Charlie, city=New York)],
   //   Paris=[Person(name=Bob, city=Paris)]
   // }

7. flatMap: Maps each element to a collection and flattens the results.
   

   val numbers = listOf(1, 2, 3)
   val expandedNumbers = numbers.flatMap { listOf(it, it * 10) }
   println(expandedNumbers) // Output: [1, 10, 2, 20, 3, 30]

8. partition: Splits the collection into two based on a predicate.

Sequences are lazily evaluated collections in Kotlin. They are useful when working with large datasets or when the operations are computationally intensive.

You can create a sequence from a collection:

val numbers = listOf(1, 2, 3, 4, 5)
val numberSequence = numbers.asSequence()

Or generate one:

val naturalNumbers = generateSequence(1) { it + 1 }
val firstTenNumbers = naturalNumbers.take(10).toList()
println(firstTenNumbers) // Output: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Example of using a sequence:

// Even though we start with a range of 1 to 1,000,000, only the necessary computations
// are performed due to lazy evaluation.
val numbers = (1..1_000_000).asSequence()
    .map { it * 2 }
    .filter { it % 3 == 0 }
    .take(10)
    .toList()
println(numbers) // Output: [6, 12, 18, 24, 30, 36, 42, 48, 54, 60]

1. Lazy Evaluation: Operations are evaluated as needed, which can improve performance.
2. Avoids Intermediate Collections: Reduces memory overhead. 

Kotlin provides collection builders for creating collections in a functional style.

val numbers = buildList {
    add(1)
    add(2)
    addAll(listOf(3, 4, 5))
}
println(numbers) // Output: [1, 2, 3, 4, 5]

Kotlin sequences are not inherently parallel. However, Kotlin coroutines can be used for asynchronous and parallel operations. For example:

import kotlinx.coroutines.*
import kotlinx.coroutines.Dispatchers

fun main() = runBlocking {
    val numbers = (1..5).toList()
    val squares = numbers.map { number ->
        async(Dispatchers.Default) {
            number * number
        }
    }.awaitAll()
    println(squares) // Output: [1, 4, 9, 16, 25]
}

Java's collection framework is robust and has evolved over time, especially with the introduction of Streams in Java 8.

1. Immutable Collections: Introduced in Java 9 with List.of(), Set.of(), Map.of().
2. Mutable Collections: The standard ArrayList, HashSet, HashMap, etc.

Example of Immutable List in Java:

List numbers = List.of(1, 2, 3);

Java 8 introduced Streams ( Also lazily loaded ) , which provide functional operations on collections.

List numbers = Arrays.asList(1, 2, 3, 4, 5);
List squares = numbers.stream()
    .map(n -> n * n)
    .collect(Collectors.toList());
System.out.println(squares); // Output: [1, 4, 9, 16, 25]

Sequences in Kotlin

1. Similar to Java Streams but more integrated into the language.
2. No Need for Explicit Conversion: Collections can be seamlessly converted to sequences.

Streams in Java

1. Separate API: Requires conversion using stream() method.
2. Terminal Operations: Must perform a terminal operation to execute the stream pipeline.

One of the key strengths of Kotlin is its seamless interoperability with Java. This means you can:

1. Call Kotlin code from Java.
2. Call Java code from Kotlin.
3. Use existing Java libraries and frameworks in Kotlin projects.
4. Migrate codebases incrementally from Java to Kotlin.

In this section, we'll explore how Kotlin and Java interoperate, covering various aspects such as calling functions, handling nullability, working with classes, exceptions, and more.

Kotlin functions can be called from Java code without any special effort. 

Kotlin Function:

// File: Utils.kt
package cc.gmem.utils

fun greet(name: String): String {
    return "Hello, $name!"
}

Calling from Java:

import cc.gmem.utils.UtilsKt;

public class Main {
    public static void main(String[] args) {
        String message = UtilsKt.greet("Alice");
        System.out.println(message); // Output: Hello, Alice!
    }
}

Explanation:

1. By default, top-level functions in Kotlin are compiled into a class named after the file with the suffix Kt.
2. In this case, the class is UtilsKt, and the function greet is a static method. 

You can customize the generated class name using the @file:JvmName annotation.

@file:JvmName("UtilFunctions")
package cc.gmem.utils

fun greet(name: String): String {
    return "Hello, $name!"
}

 For the function above you can call it from Java:

import cc.gmem.utils.UtilFunctions;

public class Main {
    public static void main(String[] args) {
        String message = UtilFunctions.greet("Alice");
        System.out.println(message);
    }
}

Kotlin's companion objects can be used to expose static members to Java.

class MathUtils {
    companion object {
        fun add(a: Int, b: Int): Int {
            return a + b
        }
    }
}

 For the function above you can call it from Java:

public class Main {
    public static void main(String[] args) {
        int sum = MathUtils.Companion.add(5, 3);
        System.out.println(sum); // Output: 8
    }
}

To make the method appear as a static method in Java, use the @JvmStatic annotation.

class MathUtils {
    companion object {
        @JvmStatic
        fun add(a: Int, b: Int): Int {
            return a + b
        }
    }
}

Calling from Java:

public class Main {
    public static void main(String[] args) {
        int sum = MathUtils.add(5, 3);
        System.out.println(sum);
    }
}

Kotlin properties are compiled to getter and setter methods in Java.

class Person(var name: String, val age: Int)

Accessing from Java:

public class Main {
    public static void main(String[] args) {
        Person person = new Person("Alice", 30);
        System.out.println(person.getName()); // Getter for 'name'
        person.setName("Bob"); // Setter for 'name'
        System.out.println(person.getAge()); // Getter for 'age' (no setter since it's 'val')
    }
}

Kotlin's null safety affects how types are represented in Java.

fun getName(): String? {
    return null
}

public class Main {
    public static void main(String[] args) {
        String name = UtilsKt.getName();
        if (name != null) {
            System.out.println(name.length());
        } else {
            System.out.println("Name is null");
        }
    }
}

Kotlin can seamlessly use existing Java classes and methods.

public class Calculator {
    public int add(int a, int b) {
        return a + b;
    }
}

Using in Kotlin:

fun main() {
    val calculator = Calculator()
    val sum = calculator.add(5, 3)
    println(sum) // Output: 8
}

public class MathUtils {
    public static int multiply(int a, int b) {
        return a * b;
    }
}

Using in Kotlin: 

fun main() {
    val product = MathUtils.multiply(4, 5)
    println(product) // Output: 20
}

When calling Java code from Kotlin, types are treated as platform types, which can be nullable or non-nullable.

public String getName() {
    return null;
}

Using in Kotlin:

fun main() {
    val name = getName()
    println(name.length) // May throw NullPointerException
    val name1: String? = getName() // Safe: Allows null
}

Since getName() is a platform type, Kotlin does not enforce null checks. It's up to the developer to handle potential nulls. 

Kotlin does not have checked exceptions. When calling Java methods that throw checked exceptions, you need to handle them manually.

public void readFile(String path) throws IOException {
    // ...
}

Using in Kotlin:

fun main() {
    try {
        readFile("file.txt")
    } catch (e: IOException) {
        e.printStackTrace()
    }
}

Please note: You need to catch the exception explicitly; the compiler does not enforce it.

Kotlin recognizes Java's nullability annotations to improve null safety.

public @Nullable String getName() {
    return null;
}

Using in Kotlin: 

fun main() {
    val name = getName()
    if (name != null) {
        println(name.length)
    } else {
        println("Name is null")
    }
}

Kotlin treats @Nullable types as nullable. 

Kotlin functions with default parameters can generate overloads for Java using @JvmOverloads.

class Greeter {
    @JvmOverloads
    fun greet(name: String = "Guest") {
        println("Hello, $name!")
    }
}

Calling from Java:

public class Main {
    public static void main(String[] args) {
        Greeter greeter = new Greeter();
        greeter.greet();          // Calls greet() with default parameter
        greeter.greet("Alice");   // Calls greet(String)
    }
}

By default, Kotlin properties are accessed via getters and setters. To expose a public field directly to Java, use @JvmField.

class Constants {
    @JvmField
    val MAX_COUNT = 100
}

 Accessing from Java:

public class Main {
    public static void main(String[] args) {
        int max = Constants.MAX_COUNT;
        System.out.println(max); // Output: 100
    }
}

Kotlin supports Java's annotation processing tools.

1. Using Annotations: You can use annotations like @Entity, @Autowired, etc., in Kotlin classes.
2. Annotation Processors: Tools like Dagger, Hibernate, and Spring Data work with Kotlin code.

Example:

@Entity
data class User(
    @Id val id: Long,
    val name: String
)

About Single Abstract Method (SAM) Interfaces:

1. In Java, functional interfaces can be implemented using lambda expressions.
2. Kotlin supports SAM conversions for Java interfaces but not for Kotlin interfaces.

public interface Runnable {
    void run();
}

Using in Kotlin: 

fun main() {
    // You can pass a lambda to a Java SAM interface in Kotlin.
    val runnable = Runnable { println("Running") }
    runnable.run()
}

Kotlin data classes generate equals(), hashCode(), toString(), and copy() methods.

Kotlin Data Class:

data class User(val name: String, val age: Int)

Using in Java:

public class Main {
    public static void main(String[] args) {
        User user = new User("Alice", 30);
        System.out.println(user.getName()); // Access properties via getters
        System.out.println(user); // Calls toString()
    }
}

Plese note that the

copy()

Kotlin's typealias declarations are not visible in Java.

typealias StringMap = Map<String, String>

fun getMap(): StringMap {
    return mapOf("key" to "value")
}

Java usage:

public class Main {
    public static void main(String[] args) {
        Map<String, String> map = UtilsKt.getMap();
        System.out.println(map);
    }
}

Kotlin extension functions are not directly accessible from Java.

Kotlin Extension Function:

fun String.isPalindrome(): Boolean {
    return this == this.reversed()
}

Using in Java:

public class Main {
    public static void main(String[] args) {
        String word = "level";
        // Extension functions are compiled into static methods with the receiver type as the first parameter
        boolean result = StringExtensionsKt.isPalindrome(word);
        System.out.println(result); // Output: true
    }
}

Kotlin coroutines can be used from Java code, but it's more complex. 

Kotlin Suspended Function:

suspend fun fetchData(): String {
    delay(1000L)
    return "Data"
}

Using in Java:

public class Main {
    public static void main(String[] args) {
        // Calling suspend functions from Java requires handling Continuation, making it less straightforward.
        Continuation continuation = new Continuation() {
            @Override
            public CoroutineContext getContext() {
                return EmptyCoroutineContext.INSTANCE;
            }

            @Override
            public void resumeWith(Object result) {
                if (result instanceof Result) {
                    Result res = (Result) result;
                    System.out.println(res.getOrNull());
                }
            }
        };

        FetchDataKt.fetchData(continuation);
    }
}

Having explored various advanced features of Kotlin, including coroutines, delegation, generics and variance, collections, and interoperability with Java, let's look at how these features are applied in real-world scenarios.

Kotlin is officially supported by Google as the preferred language for Android app development. It offers concise syntax, null safety, and powerful features that improve developer productivity.

Problem: Android apps often perform operations that can block the main thread, such as network requests or database operations. Blocking the main thread can lead to a poor user experience.

Solution with Coroutines: Use coroutines to perform asynchronous tasks without blocking the main thread.

Example:

class MainActivity : AppCompatActivity() {

    private val viewModel: DataViewModel by viewModels()

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        // Launches a coroutine tied to the lifecycle of the activity.
        lifecycleScope.launch {
            val data = viewModel.fetchData()
            updateUI(data)
        }
    }

    private fun updateUI(data: String) {
        // Update UI elements with the fetched data
    }
}

class DataViewModel : ViewModel() {
    // Switches the coroutine context to a background thread for I/O operations.
    suspend fun fetchData(): String = withContext(Dispatchers.IO) {
        // Simulate network call
        delay(1000)
        "Data from server"
    }
}

Problem: Reducing boilerplate code and improving code organization in Android applications.

Solution with Extensions and Delegation

1. Extensions: Add utility functions to existing classes without inheritance.
2. Delegation: Use property delegation for shared preferences.

Example:

// Extension function for Toast messages
fun Context.showToast(message: String, duration: Int = Toast.LENGTH_SHORT) {
    Toast.makeText(this, message, duration).show()
}

// Usage in an Activity
class MainActivity : AppCompatActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        showToast("Welcome to the app!")
    }
}

// Delegated property for SharedPreferences
class UserPreferences(context: Context) {
    private val prefs: SharedPreferences by lazy {
        context.getSharedPreferences("user_prefs", Context.MODE_PRIVATE)
    }

    var userName: String?
        get() = prefs.getString("user_name", null)
        set(value) = prefs.edit().putString("user_name", value).apply()
}

Problem: Dealing with null references and integrating Java libraries in Android apps.

Solution:

1. Utilize Kotlin's null safety features to prevent crashes.
2. Interoperate with Java libraries seamlessly.

Example:

// Using null safety
val intentData: String? = intent.getStringExtra("data")
intentData?.let {
    // Use the data safely
}

// Interoperating with a Java library
val dateFormat = SimpleDateFormat("yyyy-MM-dd", Locale.getDefault())
val date = dateFormat.parse("2023-10-01")

Ktor is an asynchronous framework for building microservices and web applications.

import io.ktor.application.*
import io.ktor.http.*
import io.ktor.response.*
import io.ktor.routing.*
import io.ktor.server.engine.*
import io.ktor.server.netty.*

fun main() {
    embeddedServer(Netty, port = 8080) {
        routing {
            get("/hello") {
                call.respondText("Hello, World!", ContentType.Text.Plain)
            }
            post("/data") {
                val data = call.receive()
                call.respondText("Received: $data", ContentType.Text.Plain)
            }
        }
    }.start(wait = true)
}

// Define the table
object Users : Table() {
    val id = integer("id").autoIncrement()
    val name = varchar("name", 50)
    val age = integer("age")
    override val primaryKey = PrimaryKey(id)
}

// Perform database operations
transaction {
    // Insert a new user
    Users.insert {
        it[name] = "Alice"
        it[age] = 30
    }

    // Query users
    val userList = Users.selectAll().map {
        it[Users.name] to it[Users.age]
    }
}

Leverage coroutines for high-throughput server applications. Example:

suspend fun handleRequest(request: Request): Response = coroutineScope {
    val data = async { fetchDataFromDatabase() }
    val computation = async { performComputation() }

    // Wait for both tasks to complete
    Response(data.await(), computation.await())
}

mockk library can be used for mocking:

class UserServiceTest {

    private val repository = mockk()
    private val userService = UserService(repository)

    //  In Kotlin, you can use backticks (``)* to enclose function names, allowing you to include spaces, 
    //  special characters, or even keywords that are normally not allowed in regular function identifiers. 
    @Test
    fun `should return user when found`() = runBlocking {
        val user = User(1, "Alice")
        coEvery { repository.findUser(1) } returns user

        val result = userService.getUser(1)

        assertEquals(user, result)
        coVerify { repository.findUser(1) }
    }
}

Gradle (build.gradle.kts)

plugins {
    id("org.springframework.boot") version "3.1.0"
    id("io.spring.dependency-management") version "1.1.0"
    kotlin("jvm") version "1.9.10"
    kotlin("plugin.spring") version "1.9.10"
    kotlin("plugin.jpa") version "1.9.10"
}

group = "cc.gmem"
version = "0.0.1-SNAPSHOT"
java.sourceCompatibility = JavaVersion.VERSION_11

repositories {
    mavenCentral()
}

dependencies {
    implementation("org.springframework.boot:spring-boot-starter-web")
    implementation("com.fasterxml.jackson.module:jackson-module-kotlin")
    implementation("org.jetbrains.kotlin:kotlin-reflect")
    implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
    // Add other dependencies as needed
    testImplementation("org.springframework.boot:spring-boot-starter-test")
    testImplementation("io.mockk:mockk:1.13.5")
}

tasks.withType {
    useJUnitPlatform()
}

Application Entry Point:

package cc.gmem.demo

import org.springframework.boot.autoconfigure.SpringBootApplication
import org.springframework.boot.runApplication

@SpringBootApplication
class DemoApplication

fun main(args: Array) {
    runApplication(*args)
}

package cc.gmem.demo.controller

import org.springframework.web.bind.annotation.GetMapping
import org.springframework.web.bind.annotation.PathVariable
import org.springframework.web.bind.annotation.RestController

data class Greeting(val id: Long, val content: String)

@RestController
class GreetingController {

    @GetMapping("/greeting/{name}")
    fun greeting(@PathVariable name: String): Greeting {
        return Greeting(id = 1, content = "Hello, $name!")
    }
}

./gradlew bootRun

Kotlin encourages constructor injection due to its concise syntax.

package cc.gmem.demo.service

import org.springframework.stereotype.Service

@Service
class UserService(private val userRepository: UserRepository) {

    fun findAllUsers(): List = userRepository.findAll()
}

Entity definition:

package cc.gmem.demo.model

import jakarta.persistence.*

@Entity
data class User(
    @Id @GeneratedValue(strategy = GenerationType.IDENTITY)
    val id: Long = 0,
    val name: String,
    val email: String
)

Repository interface:

package cc.gmem.demo.repository

import cc.gmem.demo.model.User
import org.springframework.data.jpa.repository.JpaRepository

interface UserRepository : JpaRepository<User, Long> {
    fun findByEmail(email: String): User?
}

package cc.gmem.demo.service

import cc.gmem.demo.model.User
import cc.gmem.demo.repository.UserRepository
import org.springframework.stereotype.Service

@Service
class UserService(private val userRepository: UserRepository) {

    fun getUserByEmail(email: String): User? = userRepository.findByEmail(email)
}

package cc.gmem.demo.controller

import cc.gmem.demo.model.User
import cc.gmem.demo.service.UserService
import org.springframework.web.bind.annotation.*

@RestController
@RequestMapping("/users")
class UserController(private val userService: UserService) {

    @GetMapping("/{email}")
    fun getUser(@PathVariable email: String): User? {
        return userService.getUserByEmail(email)
    }
}

package cc.gmem.demo.config

import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import cc.gmem.demo.service.CustomService

@Configuration
class AppConfig {

    @Bean
    fun customService(): CustomService = CustomService()
}

Spring Framework supports coroutines and reactive programming.

Add kotlinx-coroutines-reactor as a dependency:

implementation("org.jetbrains.kotlinx:kotlinx-coroutines-reactor")

Using suspend Functions in Controllers:

import kotlinx.coroutines.delay
import org.springframework.web.bind.annotation.GetMapping
import org.springframework.web.bind.annotation.RestController

@RestController
class AsyncController {

    @GetMapping("/async")
    suspend fun getAsyncData(): String {
        delay(1000) // Simulate non-blocking delay
        return "Async Response"
    }
}

Writing Tests with JUnit 5:

package cc.gmem.demo.service

import cc.gmem.demo.model.User
import cc.gmem.demo.repository.UserRepository
import io.mockk.*
import org.junit.jupiter.api.Assertions.*
import org.junit.jupiter.api.Test
import org.springframework.boot.test.context.SpringBootTest

@SpringBootTest
class UserServiceTest {

    private val userRepository = mockk()
    private val userService = UserService(userRepository)

    @Test
    fun `should return user when email exists`() {
        val email = "test@example.com"
        val user = User(id = 1, name = "Test User", email = email)

        every { userRepository.findByEmail(email) } returns user

        val result = userService.getUserByEmail(email)

        assertEquals(user, result)
        verify { userRepository.findByEmail(email) }
    }
}

Spring WebFlux is the reactive web framework in the Spring ecosystem, designed for building non-blocking, event-driven applications. When combined with Kotlin's coroutines, Spring WebFlux provides an elegant, efficient way to handle asynchronous operations.

In this section, we'll explore how to set up reactive routes in Kotlin using Spring WebFlux and coroutines. We'll focus on declarative routing with

coRouter

Reactive programming is a programming paradigm focused on building asynchronous, non-blocking systems that are scalable and resilient. Unlike traditional blocking I/O, reactive programming allows your application to handle a large number of requests efficiently by reacting to incoming data streams as they arrive, rather than waiting for blocking operations to complete.

Spring WebFlux is the reactive counterpart of Spring MVC, and it is built on the Project Reactor library, which provides the core reactive API.

Spring WebFlux introduces several new concepts for building reactive applications:

• Mono - Represents a single asynchronous value or an empty value.
• Flux - Represents a stream of asynchronous values, zero or more.
• Non-blocking I/O - WebFlux runs on top of a non-blocking I/O framework such as Netty or Undertow.
• Coroutines - Kotlin’s coroutines allow us to write asynchronous, non-blocking code in a declarative style.

In Kotlin, using coroutines makes reactive programming more intuitive by handling asynchronous tasks in a sequential, readable manner.

One of the core features of Spring WebFlux is the ability to define reactive routes using Kotlin DSL with

coRouter

Here’s how you can define reactive routes:

import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import org.springframework.web.reactive.function.server.coRouter

@Configuration
class RoutesConfig {

    @Bean
    fun apiRouter(handler: ApiHandler) = coRouter {
        "/api".nest {
            GET("/items", handler::getAllItems)
            GET("/items/{id}", handler::getItemById)
            POST("/items", handler::createItem)
        }
    }
}

In this example, the

coRouter

• The GET and POST methods define routes that map to specific paths, like

  /items

  and

  /items/{id}

  .

• handler::getAllItems

  refers to handler functions that will process incoming requests asynchronously.

• nest

  is used to group multiple routes under a common path, such as

  /api

  .

This approach replaces traditional annotation-based controllers with a more declarative and functional style, making the code more concise and readable.

Spring WebFlux, when used with Kotlin, leverages coroutines for asynchronous processing. This helps eliminate the complexity of callbacks and makes reactive code look more like traditional blocking code while retaining its non-blocking nature.

For instance, here’s a handler function written with coroutines:

import kotlinx.coroutines.reactor.awaitSingle
import org.springframework.web.reactive.function.server.ServerRequest
import org.springframework.web.reactive.function.server.ServerResponse

class ApiHandler(private val service: ItemService) {

    suspend fun getAllItems(request: ServerRequest): ServerResponse {
        val items = service.getAllItems().collectList().awaitSingle()
        return ServerResponse.ok().bodyValueAndAwait(items)
    }

    suspend fun getItemById(request: ServerRequest): ServerResponse {
        val id = request.pathVariable("id")
        val item = service.getItemById(id).awaitSingleOrNull()
        return item?.let { ServerResponse.ok().bodyValueAndAwait(it) }
            ?: ServerResponse.notFound().buildAndAwait()
    }

    suspend fun createItem(request: ServerRequest): ServerResponse {
        val newItem = request.awaitBody()
        val savedItem = service.saveItem(newItem).awaitSingle()
        return ServerResponse.ok().bodyValueAndAwait(savedItem)
    }
}

// Implementation of ItemService

import reactor.core.publisher.Flux
import reactor.core.publisher.Mono

data class Item(val id: String, val name: String, val price: Double)

interface ItemService {
    fun getAllItems(): Flux<Item>
    fun getItemById(id: String): Mono<Item>
    fun saveItem(item: Item): Mono<Item>
}

class ItemServiceImpl : ItemService {

    private val items = mutableListOf(
        Item("1", "Laptop", 999.99),
        Item("2", "Smartphone", 599.99),
        Item("3", "Tablet", 299.99)
    )

    override fun getAllItems(): Flux<Item> {
        return Flux.fromIterable(items)
    }

    override fun getItemById(id: String): Mono<Item> {
        val item = items.find { it.id == id }
        return if (item != null) {
            Mono.just(item)
        } else {
            Mono.empty()
        }
    }

    override fun saveItem(item: Item): Mono<Item> {
        items.add(item)
        return Mono.just(item)
    }
}

In this code:

• awaitSingle() is used to await the result of a

  Mono

  without blocking the thread.
• suspend functions are used to declare that these functions will be executed asynchronously in a non-blocking manner using coroutines.
• Handlers return a

  ServerResponse

  object, which is a reactive response object in WebFlux.

Kotlin’s coroutine support in WebFlux allows you to avoid callback hell and write non-blocking code in a sequential, easy-to-read style.

The post A Comprehensive Study of Kotlin for Java Developers appeared first on 绿色记忆.

通常我们基于Maven来开发Jenkins插件。

修改Maven配置文件，添加：

<profile>
    <id>jenkins</id>
    <repositories>
        <repository>
            <id>repo.jenkins-ci.org</id>
            <url>http://repo.jenkins-ci.org/public/</url>
        </repository>
    </repositories>
    <pluginRepositories>
        <pluginRepository>
            <id>repo.jenkins-ci.org</id>
            <url>http://repo.jenkins-ci.org/public/</url>
        </pluginRepository>
    </pluginRepositories>
</profile>

插件项目的父POM通常设置为：

<parent>
    <groupId>org.jenkins-ci.plugins</groupId>
    <artifactId>plugin</artifactId>
    <version>3.43</version>
    <relativePath />
  </parent>

此POM为构件插件提供了合理的缺省配置， 修改属性以定制配置项：

<properties>
    <jenkins.version>2.60.1</jenkins.version>
    <java.level>8</java.level>
  </properties>

这是一个Maven插件，用于构建Jenkins插件。

此目标用于创建一个新的插件项目骨架。目前已经废弃，应当考虑基于原型生成插件项目骨架

目标完整名称：org.jenkins-ci.tools:maven-hpi-plugin:3.7:hpi。构建Jenkins插件的WAR包。

此目标的必需参数：

此目标主要的可选参数：

类似hpi:hpi，但是打包为Debug布局。

目标完整名称：org.jenkins-ci.tools:maven-hpi-plugin:3.7:run。使用当前插件来运行Jenkins。

此目标的必需参数：

此目标主要的可选参数：

可以通过mvnDebug进行Jenkins插件的单步跟踪。

mvnDebug hpi:run

# 或者
export MAVEN_OPTS="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,address=8000,suspend=n"
mvn hpi:run

使用系统属性来调整Jenkins的HTTP端口，以及上下文路径：

mvn hpi:run -Djetty.port=8090  -Dhpi.prefix=/jenkins

现在可以通过Gradle JPI plugin，基于Gradle开发Jenkins插件。

plugins {
  id 'org.jenkins-ci.jpi' version '0.33.0'
}

group = 'org.jenkins-ci.plugins'
version = '1.2.0-SNAPSHOT'
description = 'A description of your plugin'

jenkinsPlugin {
    // 依赖的Jenkins核心版本
    coreVersion = '1.420'

    // 插件的标识符，默认为项目名去掉-plugin后缀
    shortName = 'hello-world'

    // 人类可读的名称
    displayName = 'Hello World plugin built with Gradle'

    // 在Jenkins Wiki上的插件页面
    url = 'http://wiki.jenkins-ci.org/display/JENKINS/SomePluginPage'

    // 可选的插件源码库地址
    gitHubUrl = 'https://github.com/jenkinsci/some-plugin'

    // 是否让插件Classloader优先于核心Classloader加载类
    pluginFirstClassLoader = true

    // 不希望看到的核心的类前缀
    maskClasses = 'groovy.grape org.apache.commons.codec'

    // 当前版本的配置和什么版本之后的兼容
    compatibleSinceVersion = '1.1.0'

    // 开发服务器的工作目录
    workDir = file('/tmp/jenkins')

    // 部署此插件到什么地方
    repoUrl = 'https://repo.jenkins-ci.org/releases'

    // 部署此插件的快照版本到什么地方调试插件
    snapshotRepoUrl = 'https://repo.jenkins-ci.org/snapshots'

    // 是否禁用测试的依赖注入，用于检查Jelly语法等
    disabledTestInjection = false

    // 本地化任务输出目录
    localizerOutputDir = "${project.buildDir}/generated-src/localizer"

    // 是否配置中心仓库、本地仓库、Jenkins中心仓库
    configureRepositories = false

    // 是否配置部署仓库
    configurePublishing = false

    // 插件扩展名，hpi或者jpi
    fileExtension = 'hpi'

    // 开发者列表
    developers {
        developer {
            id 'abayer'
            name 'Andrew Bayer'
            email 'andrew.bayer@gmail.com'
        }
    }

    // License信息
    licenses {
        license {
            name 'Apache License, Version 2.0'
            url 'https://www.apache.org/licenses/LICENSE-2.0.txt'
            distribution 'repo'
            comments 'A business-friendly OSS license'
        }
    }
}

dependencies {
    // 依赖其它插件，目标插件的类在编译期可见
    jenkinsPlugins 'org.jenkinsci.plugins:git:1.1.15'
    // 可选依赖
    optionalJenkinsPlugins 'org.jenkins-ci.plugins:ant:1.2'
    // 测试依赖
    jenkinsTest 'org.jenkins-ci.main:maven-plugin:1.480'
    // 为server任务安装额外的插件
    jenkinsServer 'org.jenkins-ci.plugins:ant:1.2'
}

plugins {
    id 'groovy'
    id "org.jenkins-ci.jpi" version "0.33.0"
}

sourceCompatibility = 1.8

group 'io.jenkins.plugins'
version '1.0-SNAPSHOT'
description = "This plugin provides pipeline steps for ALCM."

repositories {
    maven { url 'http://maven.aliyun.com/nexus/content/groups/public/' }
    maven { url 'https://repo.jenkins-ci.org/public/' }
    mavenLocal()
}

jenkinsPlugin {
    coreVersion = "2.164.1"
    displayName = "PA ALCM Steps"
    url = "http://wiki.jenkins-ci.org/display/JENKINS/ALCMSteps"
    shortName = "alcm-steps"
    workDir = file('/tmp/jenkins')
    fileExtension = 'jpi'
    pluginFirstClassLoader = true
}

dependencies {
    compile 'org.codehaus.groovy:groovy-all:2.3.11'
    compile 'cc.gmem.yun.alcm:client:1.0-SNAPSHOT'
    compile 'org.jenkins-ci.plugins:structs:1.19'
    testCompile group: 'junit', name: 'junit', version: '4.12'
    jenkinsTest 'org.jenkins-ci.plugins.workflow:workflow-step-api:2.19'
    jenkinsTest 'org.jenkins-ci.plugins.workflow:workflow-cps:2.39'
    jenkinsTest 'org.jenkins-ci.plugins.workflow:workflow-job:2.11.2'
    jenkinsTest 'org.jenkins-ci.plugins.workflow:workflow-basic-steps:2.6'
    jenkinsTest 'org.jenkins-ci.plugins.workflow:workflow-durable-task-step:2.13'
    jenkinsTest 'org.jenkins-ci.plugins.workflow:workflow-api:2.30'
    jenkinsTest 'org.jenkins-ci.plugins.workflow:workflow-support:3.3'
    jenkinsTest 'org.jenkins-ci.plugins:script-security:1.39'
    jenkinsTest 'org.jenkins-ci.plugins:scm-api:2.2.6'
} 

-Djenkins.httpPort=8082

# 调试
./gradlew server -Dorg.gradle.debug=true

# 定制端口
/gradlew server -Dorg.gradle.jvmargs=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005

# 禁用Jelly缓存
./gradlew -Dstapler.jelly.noCache=false server

Jenkins提供了一系列插件的原型，我们可以根据基于这些原型生成自己的插件的骨架代码：

mvn -U -P jenkins archetype:generate -Dfilter="io.jenkins.archetypes:"

# 非交互式
mvn archetype:generate  -P jenkins -B -DarchetypeGroupId=io.jenkins.archetypes \
    -DarchetypeArtifactId=hello-world-plugin \
    -DarchetypeVersion=1.5 \
    -DgroupId=cc.gmem.yun.alcm -DartifactId=alcm-steps -Dversion=1.0-SNAPSHOT \
    -Dpackage=cc.gmem.yun.alcm.pipeline.jenkins.steps

从列表中选择一个合适的原型，然后根据提示填写各项参数即可。

本章后续内容均针对基于hello-world-plugin生成的代码骨架。

从原型生成的插件目录结构如下：

├── alcm-steps.iml
├── pom.xml
├── src
│   ├── main
│   │   ├── java      # 插件源代码
│   │   │   └── cc
│   │   │       └── gmem
│   │   │           └── yun
│   │   │               └── alcm
│   │   │                   └── pipeline
│   │   │                       └── jenkins
│   │   │                           └── steps
│   │   │                               └── HelloWorldBuilder.java # 插件类
│   │   ├── resources  # Jelly/Groovy视图文件
│   │   │   ├── index.jelly  # 插件说明信息 
│   │   │   ├── cc
│   │   │   │   └── gmem
│   │   │   │       └── yun
│   │   │   │           └── alcm
│   │   │   │               └── pipeline
│   │   │   │                   └── jenkins
│   │   │   │                       └── steps
│   │   │   │                           ├── HelloWorldBuilder
│   │   │   │                           │   ├── config.jelly # 插件配置页面模板
│   │   │   │                           │   ├── config.properties # 模板变量值
│   │   │   │                           │   ├── config_zh_CN.properties # 模板变量值（国际化）
│   │   │   │                           │   ├── help-name.html # 字段name的帮助信息
│   │   │   │                           │   ├── help-name_zh_CN.html
│   │   │   │                           │   ├── help-useFrench.html
│   │   │   │                           │   └── help-useFrench_zh_CN.html
│   │   └── webapp  # 插件的静态资源

该插件会为Jenkins Job提供一个Step，此Step的配置界面如下：

Name字段的名称为name，点击后面的问号，显示的内容是help-name.html中的。 

此类是插件的实现。它继承Builder，实现了SimpleBuildStep。

SimpleBuildStep是诸如Builder、Publisher之类的Step，可以在构建过程的任何时机调用多次。这些Step应当遵循：

1. 不去实现BuildStep.prebuild方法，因为此方法假设了一种特定的执行顺序
2. 不去实现BuildStep.getProjectActions方法，因为如果此Step不是项目的静态配置的一部分，则它可能永不会被调用
3. 实现BuildStep.getRequiredMonitorService，且返回BuildStepMonitor.NONE，因为只对仅调用一次的Step有意义
4. 不去实现DependencyDeclarer
5. 不假设Executor.currentExecutor为非空，不使用Computer.currentComputer

从抽象类Builder继承的方法如下：

public abstract class Builder extends BuildStepCompatibilityLayer implements Describable<Builder>, ExtensionPoint {
    
    public boolean prebuild(Build build, BuildListener listener) {
        return true;
    }

    // 由于Builder通常不依赖于它前面的步骤的结果，因此默认返回NONE
    public BuildStepMonitor getRequiredMonitorService() {
        return BuildStepMonitor.NONE;
    }

    public Descriptor<Builder> getDescriptor() {
        return Jenkins.getInstance().getDescriptorOrDie(getClass());
    }

    public static DescriptorExtensionList<Builder,Descriptor<Builder>> all() {
        return Jenkins.getInstance().<Builder,Descriptor<Builder>>getDescriptorList(Builder.class);
    }
}

HelloWorldBuilder是插件的实现，实现插件不需要继承plugin类（这种方式已经废弃），推荐的方式是实现扩展点，并通过@hudson.Extension注解来注册（以便Jenkins自动发现）。

关于这种扩展的方式，需要注意：

1. 一般来说，你的插件应该继承已有的扩展点（实现ExtensionPoint的类），并在静态内部类中继承对应的描述符类，这些描述符类是hudson.model.Descriptor的子类
2. 注解@Extension必须放在上述内部类之上，这样Jenkins才能发现扩展

package cc.gmem.yun.alcm.pipeline.jenkins.steps;

import hudson.Launcher;
import hudson.Extension;
import hudson.FilePath;
import hudson.util.FormValidation;
import hudson.model.AbstractProject;
import hudson.model.Run;
import hudson.model.TaskListener;
import hudson.tasks.Builder;
import hudson.tasks.BuildStepDescriptor;
import org.kohsuke.stapler.DataBoundConstructor;
import org.kohsuke.stapler.QueryParameter;

import javax.servlet.ServletException;
import java.io.IOException;
import jenkins.tasks.SimpleBuildStep;
import org.jenkinsci.Symbol;
import org.kohsuke.stapler.DataBoundSetter;

public class HelloWorldBuilder extends Builder implements SimpleBuildStep {

    private final String name;
    private boolean useFrench;

    // 绑定对象时使用此构造器
    @DataBoundConstructor
    public HelloWorldBuilder(String name) {
        this.name = name;
    }

    public String getName() {
        return name;
    }

    public boolean isUseFrench() {
        return useFrench;
    }

    @DataBoundSetter
    public void setUseFrench(boolean useFrench) {
        this.useFrench = useFrench;
    }

    /**
     * SimpleBuildStep的核心方法，执行此Step
     * @param run 此Step所属的Build
     * @param workspace 此Build的工作区，用于文件系统操作
     * @param launcher 用于启动进程
     * @param listener 用于发送输出
     * @throws InterruptedException 如果此Step被中断
     * @throws IOException 出现其它错误，更“礼貌”的错误是AbortException
     */
    public void perform(Run<?, ?> run, FilePath workspace, Launcher launcher, TaskListener listener) throws InterruptedException, IOException {
        if (useFrench) {
            listener.getLogger().println("Bonjour, " + name + "!");
        } else {
            listener.getLogger().println("Hello, " + name + "!");
        }
    }

    // 此注解为Jenkins扩展定义唯一的标识符，驼峰式大小写，尽量简短。在流水线脚本中引用此Step时即使用此标识符
    // 标识符不需要全局唯一，只需要在一个扩展点内是唯一的即可
    @Symbol("greet")
    // 标记字段、方法或类，以便Huson能自动发现定位到ExtensionPoint的实现
    @Extension
    // Build / Publisher的描述符
    // Descriptor是可配置实例（Describable）的元数据，也作为Describable的工厂
    public static final class DescriptorImpl extends BuildStepDescriptor<Builder> {

        // 校验Project配置表单中的字段
        // 下面的方法检查name字段，参数value注入name的值，可以用@QueryParameter注入其它参数
        // 如果要校验useFrench字段，则需要实现doCheckUsdeFrench方法
        public FormValidation doCheckName(@QueryParameter String value, @QueryParameter boolean useFrench)
                throws IOException, ServletException {
            if (value.length() == 0)
                // 引用资源束
                return FormValidation.error(Messages.HelloWorldBuilder_DescriptorImpl_errors_missingName());
            if (value.length() < 4)
                return FormValidation.warning(Messages.HelloWorldBuilder_DescriptorImpl_warnings_tooShort());
            if (!useFrench && value.matches(".*[éáàç].*")) {
                return FormValidation.warning(Messages.HelloWorldBuilder_DescriptorImpl_warnings_reallyFrench());
            }
            return FormValidation.ok();
        }

        // 判断当前Step是否能用于目标类型的项目
        @Override
        public boolean isApplicable(Class<? extends AbstractProject> aClass) {
            return true;
        }

        // 此Step的显示名称
        @Override
        public String getDisplayName() {
            return Messages.HelloWorldBuilder_DescriptorImpl_DisplayName();
        }

    }

}

此文件是Project配置页面的模板：

<?jelly escape-by-default='true'?>
<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout" xmlns:t="/lib/hudson" xmlns:f="/lib/form">
    <!-- 文本表单字段，${%引用资源束中的条目 -->
    <f:entry title="${%Name}" field="name">
        <f:textbox />
    </f:entry>
    <f:advanced>
        <f:entry title="${%French}" field="useFrench"
                 description="${%FrenchDescr}">
            <f:checkbox />
        </f:entry>
    </f:advanced>
</j:jelly>

模板使用的资源束为config.properties。 

这个类包含了单元测试：

package cc.gmem.yun.alcm.pipeline.jenkins.steps;

import hudson.model.FreeStyleBuild;
import hudson.model.FreeStyleProject;
import hudson.model.Label;
import org.jenkinsci.plugins.workflow.cps.CpsFlowDefinition;
import org.jenkinsci.plugins.workflow.job.WorkflowJob;
import org.jenkinsci.plugins.workflow.job.WorkflowRun;
import org.junit.Rule;
import org.junit.Test;
import org.jvnet.hudson.test.JenkinsRule;

public class HelloWorldBuilderTest {

    // 标记此字段为TestRule
    // JUnit的TestRule决定一个或一组测试方法如何运行、如何报告
    @Rule
    public JenkinsRule jenkins = new JenkinsRule();

    final String name = "Bobby";

    @Test
    public void testConfigRoundtrip() throws Exception {
        // 启动一个临时Jenkins服务器，并创建自由项目
        FreeStyleProject project = jenkins.createFreeStyleProject();
        // 添加一个构建步骤
        project.getBuildersList().add(new HelloWorldBuilder(name));
        // 加载配置页面，不经修改的提交
        project = jenkins.configRoundtrip(project);
        // 判断相等
        jenkins.assertEqualDataBoundBeans(new HelloWorldBuilder(name), project.getBuildersList().get(0));
    }

    @Test
    public void testConfigRoundtripFrench() throws Exception {
        FreeStyleProject project = jenkins.createFreeStyleProject();
        HelloWorldBuilder builder = new HelloWorldBuilder(name);
        builder.setUseFrench(true);
        project.getBuildersList().add(builder);
        project = jenkins.configRoundtrip(project);

        HelloWorldBuilder lhs = new HelloWorldBuilder(name);
        lhs.setUseFrench(true);
        jenkins.assertEqualDataBoundBeans(lhs, project.getBuildersList().get(0));
    }

    @Test
    public void testBuild() throws Exception {
        FreeStyleProject project = jenkins.createFreeStyleProject();
        HelloWorldBuilder builder = new HelloWorldBuilder(name);
        project.getBuildersList().add(builder);
        // 执行构建，并断言构建成功
        FreeStyleBuild build = jenkins.buildAndAssertSuccess(project);
        // 断言本次构建的控制台输出包括指定字样
        jenkins.assertLogContains("Hello, " + name, build);
    }

    @Test
    public void testBuildFrench() throws Exception {

        FreeStyleProject project = jenkins.createFreeStyleProject();
        HelloWorldBuilder builder = new HelloWorldBuilder(name);
        builder.setUseFrench(true);
        project.getBuildersList().add(builder);

        FreeStyleBuild build = jenkins.buildAndAssertSuccess(project);
        jenkins.assertLogContains("Bonjour, " + name, build);
    }

    // 在流水线中使用此Step
    @Test
    public void testScriptedPipeline() throws Exception {
        String agentLabel = "my-agent";
        // 在本机创建一个新的Slave（Agent），等待其上线
        jenkins.createOnlineSlave(Label.get(agentLabel));
        // 定义一个流水线任务
        WorkflowJob job = jenkins.createProject(WorkflowJob.class, "test-scripted-pipeline");
        // 流水线定义，注意通过@Symbol指定的标识符来引用本插件定义的Step
        String pipelineScript
                = "node {\n"
                + "  greet '" + name + "'\n"
                + "}";
        // CPS，即continuation passing style，是运行Groovy脚本的一种方式
        // 允许应用程序被随时暂停、重启继续
        job.setDefinition(new CpsFlowDefinition(pipelineScript, true));
        WorkflowRun completedBuild = jenkins.assertBuildStatusSuccess(job.scheduleBuild2(0));
        String expectedString = "Hello, " + name + "!";
        jenkins.assertLogContains(expectedString, completedBuild);
    }

}

Jenkins插件的依赖注入，可以基于Google Guice（一个轻量级的IoC框架）实现。下面是一个例子：

// 需要被注入的接口及其实现
public interace MySvc {
    public void myServiceMethod()
}

public class MySvcImpl implements MySvc {
   public void myServiceMethod() {
       System.out.println("It works!");
   }
}

// IoC配置类
public class MyGuiceModule extends com.google.inject.AbstractModule {
    @Override
    public void configure() {
        bind(MySvc.class).to(MySvcImpl.class).in(com.google.inject.Singleton.class);
    }
}

下面的插件类会被注入MySvc：

import com.google.inject.Guice;
import com.google.inject.Inject; 
 
public class MyPublisher extends Publisher {
    private transient MySvc mySvc;

    private String someJobConfig
 
    @DataBoundConstructor
    public MyPublisher(String someJobConfig) {
        this.someJobConfig = someJobConfig;
    }
 
 
    @Inject
    public void setMySvc(MySvc mySvc) {
        this.mySvc = mySvc;
    }
 
 
    @Override
    public boolean perform(AbstractBuild<?, ?> build, Launcher launcher, BuildListener listener) {
        if (mySvc == null) {
            // 进行注入
            Guice.createInjector(new MyModule()).injectMembers(this);
        }
        mySvc.myServiceMethod();
    }
}

每当构建时，

perform(Build, Launcher, BuildListener)

launcher.launch(cmd, env, out, workDir)

// 启动进程
launcher.launch("dir", new String[0], listener.getLogger(), build.getProject().getWorkspace());


// 获取返回值
Proc proc = launcher.launch("dir", build.getEnvVars(), listener.getLogger(), build.getProject().getWorkspace());
int exitCode = proc.join();

Jenkins使用一种分布式机制来运行任务 —— 运行在Master节点上的线程，能够发送闭包到远程机器上，并且在闭包执行完毕后，取回结果。

支持分布式执行的闭包示例如下： 

private static class GetSystemProperties extends MasterToSlaveCallable<Properties,RuntimeException> {
    public Properties call() {
        return System.getProperties();
    }
    private static final long serialVersionUID = 1L;
}

关键之处在于需要实现hudson.remoting.Callable，此接口声明了返回值、异常类型。

下面的调用将闭包发送给Slave执行：

Properties systemProperties = channel.call(new GetSystemProperties());

Jenkins提供了一些关键的抽象，将其分布式执行的特性隐藏起来。

你需要把图片放在src/main/webapp下。在jelly或者Java页面上获取图片的方法是：

public String getIconPath() {
  PluginWrapper wrapper = Hudson.getInstance().getPluginManager().getPlugin([YOUR-PLUGIN-MAIN-CLASS].class);
  return Hudson.getInstance().getRootUrl() + "plugin/"+ wrapper.getShortName()+"/";
}

要在jelly页面添加空格，只需要： 

<st:nbsp/>

本章分析一些真实的Jenkins插件的代码。

Publisher可以在构建完成后，触发任意的动作。Recorder可以为每次构建记录一些统计信息。

Discard Old Build plugin是这种扩展的例子，用于在构建完毕后，检查并删除过期的构建历史。本节分析其代码：

package org.jenkinsci.plugins.discardbuild;

import hudson.Extension;
import hudson.Launcher;
import hudson.model.*;
import hudson.tasks.BuildStepDescriptor;
import hudson.tasks.BuildStepMonitor;
import hudson.tasks.Publisher;
import hudson.tasks.Recorder;
import hudson.util.RunList;
import org.kohsuke.stapler.DataBoundConstructor;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Calendar;
import java.util.HashSet;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class DiscardBuildPublisher extends Recorder {
    // 插件参数
    /**
     * If not -1, history is only kept up to this days.
     */
    private final int daysToKeep;

    // 从表单绑定插件参数
    @DataBoundConstructor
    public DiscardBuildPublisher( String daysToKeep) {
        this.daysToKeep = parse(daysToKeep);
    }


    private void deleteOldBuildsByDays(AbstractBuild<?, ?> build, BuildListener listener, int daysToKeep) {
        ArrayList<Run<?, ?>> list = updateBuildsList(build, listener);
        try {
            Calendar cal = getCurrentCalendar();
            cal.add(Calendar.DAY_OF_YEAR, -daysToKeep);
            for (Run<?, ?> r : list) {
                if (r.getTimestamp().before(cal)) {
                    discardBuild(r, "it is older than daysToKeep", listener); //$NON-NLS-1$
                }
            }
        } catch (IOException e) {
            e.printStackTrace(listener.error("")); //$NON-NLS-1$
        }
    }


    private ArrayList<Run<?, ?>> updateBuildsList(AbstractBuild<?, ?> build, BuildListener listener) {
        RunList<Run<?, ?>> builds = new RunList<Run<?, ?>>();
        ArrayList<Run<?, ?>> list = new ArrayList<Run<?, ?>>();
        // 获取本次Build对应的Job
        Job<?, ?> job = (Job<?, ?>) build.getParent();
        // 获取Job的所有构建
        builds = (RunList<Run<?, ?>>) job.getBuilds();
        list = discardLastBuilds(build, listener, builds);
        return list;
    }

    @Override
    public boolean perform(AbstractBuild<?, ?> build, Launcher launcher, BuildListener listener) {
        listener.getLogger().println("Discard old builds..."); //$NON-NLS-1$
        deleteOldBuildsByDays(build, listener, daysToKeep);
        return true;
    }

    private void discardBuild(Run<?, ?> history, String reason, BuildListener listener) throws IOException {
        listener.getLogger().printf("#%d is removed because %s\n", history.getNumber(), reason); //$NON-NLS-1$
        history.delete();
    }


    @Override
    public DescriptorImpl getDescriptor() {
        return (DescriptorImpl) super.getDescriptor();
    }

    // 描述符
    @Extension
    public static final class DescriptorImpl extends BuildStepDescriptor<Publisher> {

        @SuppressWarnings("rawtypes")
        public boolean isApplicable(Class<? extends AbstractProject> aClass) {
            return true;
        }

        /**
         * This human readable name is used in the configuration screen.
         */
        public String getDisplayName() {
            return Messages.DiscardHistoryBuilder_description();
        }
    }

    public BuildStepMonitor getRequiredMonitorService() {
        return BuildStepMonitor.NONE;
    }
}

这个扩展点用于触发某个Job的构建。

Files Found Trigger是这种扩展的例子，能够轮询一个或多个目录，当发现特定的文件后，自动触发构建。本节分析其代码。

package hudson.plugins.filesfoundtrigger;

import static hudson.Util.fixNull;

import java.io.IOException;
import java.text.MessageFormat;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.atomic.AtomicLong;
import java.util.logging.Level;
import java.util.logging.Logger;

import org.apache.commons.lang.builder.ToStringBuilder;
import org.apache.commons.lang.builder.ToStringStyle;
import org.kohsuke.stapler.DataBoundConstructor;

import com.google.common.collect.ImmutableList;
import com.thoughtworks.xstream.converters.Converter;
import com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider;
import com.thoughtworks.xstream.mapper.Mapper;

import antlr.ANTLRException;
import hudson.Extension;
import hudson.model.BuildableItem;
import hudson.model.Item;
import hudson.triggers.Trigger;
import hudson.triggers.TriggerDescriptor;
import hudson.util.RobustReflectionConverter;

public final class FilesFoundTrigger extends Trigger<BuildableItem> {

  private static final Logger LOGGER = Logger.getLogger(FilesFoundTrigger.class.getName());

  private static final AtomicLong logCounter = new AtomicLong();

  // 从什么Slave节点上检查文件是否存在，如果为空则检查master节点
  private final String node;

  // 检查的目录
  private final String directory;

  // 期望存在的文件的pattern
  private final String files;

  // 忽略的文件的pattern
  private final String ignoredFiles;

  // 仅仅当发现文件数量大于此数字时才触发
  private final String triggerNumber;

  // 额外的文件pattern
  private final ArrayList<FilesFoundTriggerConfig> additionalConfigs;

  /**
   * 构造函数
   * 
   * @param spec
   *          轮询文件系统的Cron表达式
   * @param configs
   *          文件模式列表
   * @throws ANTLRException
   *           无法解析Cron表达式抛出此异常
   */
  @DataBoundConstructor
  public FilesFoundTrigger(String spec, List<FilesFoundTriggerConfig> configs) throws ANTLRException {
    // Trigger天生基于Cron进行检查
    super(spec);

    ArrayList<FilesFoundTriggerConfig> configsCopy = new ArrayList<FilesFoundTriggerConfig>(fixNull(configs));
    FilesFoundTriggerConfig firstConfig;
    if (configsCopy.isEmpty()) {
      firstConfig = new FilesFoundTriggerConfig(null, "", "", "", "1");
    } else {
      firstConfig = configsCopy.remove(0);
    }
    this.node = firstConfig.getNode();
    this.directory = firstConfig.getDirectory();
    this.files = firstConfig.getFiles();
    this.ignoredFiles = firstConfig.getIgnoredFiles();
    this.triggerNumber = firstConfig.getTriggerNumber();
    if (configsCopy.isEmpty()) {
      configsCopy = null;
    }
    this.additionalConfigs = configsCopy;
  }


  public List<FilesFoundTriggerConfig> getConfigs() {
    ImmutableList.Builder<FilesFoundTriggerConfig> builder = ImmutableList.builder();
    builder.add(new FilesFoundTriggerConfig(node, directory, files, ignoredFiles, triggerNumber));
    if (additionalConfigs != null) {
      builder.addAll(additionalConfigs);
    }
    return builder.build();
  }

  // Cron每次触发的逻辑
  @Override
  public void run() {
    long counter = logCounter.incrementAndGet();
    for (FilesFoundTriggerConfig config : getConfigs()) {
      FilesFoundTriggerConfig expandedConfig = config.expand();
      try {
        FileSearch.Result result = FileSearch.perform(expandedConfig);
        int triggerNumber = Integer.parseInt(expandedConfig.getTriggerNumber());
        boolean triggerBuild = result.files.size() >= triggerNumber;
        if (triggerBuild) {
          // 触发构建
          job.scheduleBuild(0, new FilesFoundTriggerCause(expandedConfig));
          return;
        }
      } catch (NumberFormatException e) {
        LOGGER.log(Level.FINE, "{0} - Result: Invalid trigger number (build not triggered)", counter);
      } catch (InterruptedException e) {
        LOGGER.log(Level.FINE, "{0} - Result: Thread interrupted (build not triggered)", counter);
        Thread.currentThread().interrupt();
      }
    }
  }


  // 将当前插件注册为Trigger扩展
  @Extension
  public static final class DescriptorImpl extends TriggerDescriptor {

    @Override
    public boolean isApplicable(Item item) {
      return item instanceof BuildableItem;
    }

    @Override
    public String getDisplayName() {
      return Messages.DisplayName();
    }
  }
}

Jenkins提供一系列JUnit周边工具来简化单元测试。支持的特性包括：

1. 启动内嵌的Servlet容器，允许通过浏览器访问Jenkins页面，进行测试
2. HtmlUnit用于简化UI测试
3. 为每个测试用例准备隔离的Jenkins实例
4. 测试代码可以直接访问Jenkins对象模型，并可直接执行一些操作，而不需要通过浏览器
5. 基于注解的、声明式的测试用例环境说明

下面的例子示例了如何基于JenkinsRule进行单元测试：

import org.jvnet.hudson.test.JenkinsRule;
import org.apache.commons.io.FileUtils;
import hudson.model.*;
import hudson.tasks.Shell;
import org.junit.Test;
import org.junit.Rule;
public class AppTest {
  @Rule public JenkinsRule j = new JenkinsRule();
  @Test public void first() throws Exception {
    // 自由风格项目
    FreeStyleProject project = j.createFreeStyleProject();
    // 添加一个构建步骤
    project.getBuildersList().add(new Shell("echo hello"));
    // 得到某个构建步骤
    Shell shell = project.getBuildersList().get(Shell.class);
    // 调度构建并获得结果
    FreeStyleBuild build = project.scheduleBuild2(0).get();
    System.out.println(build.getDisplayName() + " completed");
    // 读取日志文件内容
    String s = FileUtils.readFileToString(build.getLogFile());
    assertThat(s, contains("+ echo hello"));
  }
}

某些情况下，你希望在不依赖于完整Jenkins实例的前提下进行快速测试，这时可以使用仿冒：

AbstractBuild build = Mockito.mock(AbstractBuild.class);
Mockito.when(build.getResult()).thenReturn(Result.FAILURE);

用于测试Jenkins生成的HTML页面：

HtmlPage page = j.createWebClient().goTo("computer/test/");
HtmlElement navbar = page.getElementById("left-top-nav");
assertEquals(1,navbar.selectNodes(".//a").size());

HtmlPage configPage = j.createWebClient().goTo("configure");
HtmlForm form = configPage.getFormByName("config");
form.submit((HtmlButton)last(form.getHtmlElementsByTagName("button")));

public void setEnvironmentVariables() throws IOException {
    EnvironmentVariablesNodeProperty prop = new EnvironmentVariablesNodeProperty();
    EnvVars envVars = prop.getEnvVars();
    envVars.put("sampleEnvVarKey", "sampleEnvVarValue");
    j.jenkins.getGlobalNodeProperties().add(prop);
}

下面的代码允许任何用户名登陆，只要输入的密码和用户名一致：

j.jenkins.setSecurityRealm(j.createDummySecurityRealm());

要实现一个一次性的Builder，可以选择继承TestBuilder：

FreeStyleProject project = j.createFreeStyleProject();
project.getBuildersList().add(new TestBuilder() {
    public boolean perform(AbstractBuild<?, ?> build, Launcher launcher, BuildListener listener) throws InterruptedException, IOException {
        // 向工作区中的文件中写入数据
        build.getWorkspace().child("abc.txt").write("hello","UTF-8");
        return true;
    }
});
 
project.scheduleBuild2(0);

Jenkins中构建时异步发生的，如果需要在测试代码中进行同步，可以使用OneShotEvent：

final OneShotEvent buildStarted = new OneShotEvent();
 
FreeStyleProject project = j.createFreeStyleProject();
project.getBuildersList().add(new TestBuilder() {
    public boolean perform(AbstractBuild<?, ?> build, Launcher launcher,
        BuildListener listener) throws InterruptedException, IOException {
        // 构建真正开始后，发起信号
        buildStarted.signal();
        return true;
    }
});
 
project.scheduleBuild2(0);
// 阻塞直到信号到达
buildStarted.block(); 

如果希望使用其它Jenkins插件提供的类，只需要将目标插件声明为Maven依赖即可。很多插件专门提供基础功能，供其它插件依赖，例如Git Client Plugin。

应当尽量依赖插件（而不是自己引入jar依赖），因为hpi打包时，依赖的插件（以及这些插件的传递性依赖）会自动排除，可以减小压缩包的大小。

共享库没有ClassLoader隔离机制，Jenkins Core的同名类会优先使用，从而导致冲突。

解决办法是，将存在冲突的部分代码封装到插件中，然后共享库调用插件。

插件具有类隔离机制，通过适当的配置即可避免和Jenkins Core的类冲突。但是，在基于JenkinsRule进行测试时，这种类隔离机制不生效，需要修改JenkinsRule的源码。

The post Jenkins插件开发 appeared first on 绿色记忆.

OpenAPI是一套API规范（ OpenAPI Specification ，OAS），用于定义RESTful API的接口。OpenAPI最初来自SmartBear的Swagger规范。

OpenAPI 目前的版本是3.0，当前Swagger和OpenAPI的关系是：

1. OpenAPI是一套规范
2. Swagger是实现OpenAPI规范的工具集，包括：
   1. Swagger Editor：允许你使用YAML语言在浏览器中编写规范，并实时查看生成的API文档
   2. Swagger UI：从OAS兼容的API动态生成一套Web的美观的API文档
   3. Swagger Codegen：用于生成OpenAPI的客户端库（SDK）、服务器桩代码、文档
   4. Swagger Parser：从Java解析Open API定义的独立库
   5. Swagger Core：一个Java库，用于创建、消费、使用OpenAPI定义
   6. Swagger Inspector：从现有的API生成OpenAPI定义、验证API的测试工具
   7. SwaggerHub：OpenAPI的API设计、文档平台

Swagger并非唯一支持OpenAPI的工具，到OpenAPI-Specification的GitHub页面可以看到相关工具的列表。

{
  // 基本信息
  "swagger": "2.0",
  "info": {
    "title": "Kubernetes",
    "version": "v1.12.1"
  },
  // API端点列表
  "paths": {
    // 端点
    "/api/": {
      // 操作
      "get": {
        "description": "get available API versions",
        // 支持的MIME类型
        "consumes": [
          "application/json",
          "application/yaml",
          "application/vnd.kubernetes.protobuf"
        ],
        "produces": [
          "application/json",
          "application/yaml",
          "application/vnd.kubernetes.protobuf"
        ],
        // 支持的协议
        "schemes": [
          "https"
        ],
        "tags": [
          "core"
        ],
        // 操作的唯一标识
        "operationId": "getCoreAPIVersions",
        // 请求参数说明
        "parameters": [
          {
            "type": "string",
            "description": "Username ",
            "name": "username",
            // 这是URL路径变量
            "in": "path",
            "required": true
          },
          {
            "name": "user",
            // 这是请求体参数，通常是映射到模型的JSON
            "in": "body",
            "required": true,
            "schema": {
              "type": "object",
              "$ref": "#/definitions/models.User"
            }
          },
          {
            "type": "integer",
            "name": "size",
            // 这是请求参数
            "in": "query"
          }
        ],
        // 响应说明，每个状态码对应一个元素
        "responses": {
          "200": {
            "description": "OK",
            // 响应的Schema
            "schema": {
              "$ref": "#/definitions/io.k8s.apimachinery.pkg.apis.meta.v1.APIVersions"
            }
          },
          "401": {
            "description": "Unauthorized"
          }
        }
      }
    }
  }
  // Schema定义列表
  "definitions": {
    // Schema名以域名倒写形式
    "io.k8s.apimachinery.pkg.apis.meta.v1.APIVersions": {
      "description": "APIVersions lists the versions that are available, to allow clients to discover the API at /api, which is the root path of the legacy v1 API.",
      // 必须属性
      "required": [
        "versions",
        "serverAddressByClientCIDRs"
      ],
      // 属性规格列表
      "properties": {
        "apiVersion": {
          "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources",
          "type": "string"
        },
        "kind": {
          "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds",
          "type": "string"
        },
        "serverAddressByClientCIDRs": {
          "description": "a map of client CIDR to server address that is serving this group. This is to help clients reach servers in the most network-efficient way possible. Clients can use the appropriate server address as per the CIDR that they match. In case of multiple matches, clients should use the longest matching CIDR. The server returns only those CIDRs that it thinks that the client can match. For example: the master will return an internal IP CIDR only, if the client reaches the server using an internal IP. Server looks at X-Forwarded-For header or X-Real-Ip header or request.RemoteAddr (in that order) to get the client IP.",
          "type": "array",
          "items": {
            "$ref": "#/definitions/io.k8s.apimachinery.pkg.apis.meta.v1.ServerAddressByClientCIDR"
          }
        },
        "versions": {
          "description": "versions are the api versions that are available.",
          "type": "array",
          "items": {
            "type": "string"
          }
        }
      },
      // K8S扩展字段
      "x-kubernetes-group-version-kind": [
        {
          "group": "",
          "kind": "APIVersions",
          "version": "v1"
        }
      ]
    }
  }
}

这是一个Node.js开发的工具，可以将Swagger 2.0转换为OAS 3.0，执行下面的命令安装：

npm install -g swagger2openapi

调用格式：

swagger2openapi source-spec.json [options]

常用选项说明：

--resolveInternal    # 是否解析内部引用
--warnProperty       # 警告扩展配置，默认x-s2o-warning
-e, --encoding       # 输入输出编码，默认utf8
-f, --fatal          # 如果引用解析失败，则终止转换
-i, --indent         # JSON缩进，默认4
-o, --outfile        # 输出到文件而非标准输出
-p, --patch          # 尝试修复源定义中的错误
-r, --resolve        # 是否解析外部引用
-t, --targetVersion  # OAS版本，默认3.0.0
-u, --url            # 源的URL
-w, --warnOnly       # 遇到不可修复错误时发出警告而非报错
-y, --yaml           # 输出为YAML格式而非JSON

通过某些工具，可以从既有代码中生成Swagger API文档。

该工具能够将Go Annotations转换为Swagger 2.0文档，为多种流行的Go Web框架（例如Gin）提供了插件，从而快速和既有Web项目集成。

执行下面的命令安装到GOPATH下：

go get -u github.com/swaggo/swag/cmd/swag

在项目根目录，使用

swag init

-g

swag init -g pkg/route/routers.go

使用swag init命令之后，你需要导入生成的docs包以及swaggo的另外两个包：

_ "github.com/gmemcc/myproject/docs"
import "github.com/swaggo/gin-swagger" // gin-swagger middleware
import "github.com/swaggo/files" // swagger embed files

General API annotations可用字段：

// @title Swagger Example API
// @version 1.0
// @description This is a sample server celler server.
// @termsOfService http://swagger.io/terms/

// @contact.name API Support
// @contact.url http://www.swagger.io/support
// @contact.email support@swagger.io

// @license.name Apache 2.0
// @license.url http://www.apache.org/licenses/LICENSE-2.0.html

// @host localhost:8080
// @BasePath /api/v1
// @query.collection.format multi

// @securityDefinitions.basic BasicAuth

// @securityDefinitions.apikey ApiKeyAuth
// @in header
// @name Authorization

// @securitydefinitions.oauth2.application OAuth2Application
// @tokenUrl https://example.com/oauth/token
// @scope.write Grants write access
// @scope.admin Grants read and write access to administrative information

// @securitydefinitions.oauth2.implicit OAuth2Implicit
// @authorizationurl https://example.com/oauth/authorize
// @scope.write Grants write access
// @scope.admin Grants read and write access to administrative information

// @securitydefinitions.oauth2.password OAuth2Password
// @tokenUrl https://example.com/oauth/token
// @scope.read Grants read access
// @scope.write Grants write access
// @scope.admin Grants read and write access to administrative information

// @securitydefinitions.oauth2.accessCode OAuth2AccessCode
// @tokenUrl https://example.com/oauth/token
// @authorizationurl https://example.com/oauth/authorize
// @scope.admin Grants read and write access to administrative information

// @x-extension-openapi {"example": "value on a json format"}

示例：

// @BasePath /myproject/apis/v2
// @version 2.0.0
// @title Myproject API
// @description my project
// @contact.name alex
// @contact.email myproject@gmem.cc
// @license.name Apache 2.0
// @license.url http://www.apache.org/licenses/LICENSE-2.0.html

swag init生成的docs包，导出了变量

SwaggerInfo

package main

import (
	"github.com/gin-gonic/gin"
	"github.com/swaggo/files"
	"github.com/swaggo/gin-swagger"
	
	"./docs" // docs is generated by Swag CLI, you have to import it.
)

func main() {
	// programmatically set swagger info
	docs.SwaggerInfo.Title = "Swagger Example API"
	docs.SwaggerInfo.Description = "This is a sample server Petstore server."
	docs.SwaggerInfo.Version = "1.0"
	docs.SwaggerInfo.Host = "petstore.swagger.io"
	docs.SwaggerInfo.BasePath = "/v2"
	docs.SwaggerInfo.Schemes = []string{"http", "https"}

为了在当前Web服务（的HTTP服务器）中查看API文档，需要注册路由，以gin为例： 

r := gin.New()

	// use ginSwagger middleware to serve the API docs
	r.GET("/swagger/*any", ginSwagger.WrapHandler(swaggerFiles.Handler))

	r.Run()
}

要生成API，你需要在控制器代码中添加 API Operation annotations。这些注解需要加在controller代码中，所谓controller代码，就是包含所有路由处理函数的包，以gin为例：

package controller

import (
	"fmt"
	"net/http"
	"strconv"

	"github.com/gin-gonic/gin"
	"github.com/swaggo/swag/example/celler/httputil"
	"github.com/swaggo/swag/example/celler/model"
)

// 下面就是一份API Operation annotations
// ShowAccount godoc
// @Summary Show a account
// @Description get string by ID
// @ID get-string-by-int
// @Accept  json
// @Produce  json
// @Param id path int true "Account ID"
// @Success 200 {object} model.Account
// @Header 200 {string} Token "qwerty"
// @Failure 400,404 {object} httputil.HTTPError
// @Failure 500 {object} httputil.HTTPError
// @Failure default {object} httputil.DefaultError
// 这个仅仅影响生成的Swagger API文档，你还需要调用gin的接口，注册下面这个处理函数（控制器）的路由
// @Router /accounts/{id} [get]
func (c *Controller) ShowAccount(ctx *gin.Context) {
	id := ctx.Param("id")
	aid, err := strconv.Atoi(id)
	if err != nil {
		httputil.NewError(ctx, http.StatusBadRequest, err)
		return
	}
	account, err := model.AccountOne(aid)
	if err != nil {
		httputil.NewError(ctx, http.StatusNotFound, err)
		return
	}
	ctx.JSON(http.StatusOK, account)
}

这些注解仅仅影响生成的Swagger API文档，不会对Web服务的逻辑产生任何影响。

运行Web服务，可以在http://localhost:port/swagger/index.html查看Swagger 2.0 API文档。访问/swagger/doc.json可以获得JSON格式的API

OAS是REST API的描述格式，在一个OpenAPI文件中，你可以定义完整的接口规格，包括：

1. 可用API端点列表，针对每个端点允许的操作
2. 操作的输入、输出参数
3. 身份验证方法
4. 附属信息，包括使用条款、License

关于OAS的编写，需要注意：

1. 可以用YAML、JSON两种格式编写
2. 所有关键字都是大小写敏感的

# OpenAPI的版本，可用版本  3.0.0, 3.0.1, 3.0.2
openapi: 3.0.0
info:
  title: '标题'
  description: '描述'
  version: '此OAS的版本，示例0.1.9'

# 提供此API的服务器地址列表
servers:
  - url: http://api.example.com/v1
    description: Optional server description, e.g. Main (production) server
  - url: http://staging-api.example.com
    description: Optional server description, e.g. Internal staging server for testing

# API 端点列表
paths:
  # 端点 /users
  /users:
    # 端点/users的GET方法
    get:
      # 描述性信息
      summary: Returns a list of users.
      description: Optional extended description in CommonMark or HTML.
      # 响应规格说明
      responses:
        # 200响应说明
        '200':    # status code
          description: A JSON array of user names
          # 200响应是JSON格式
          content:
            application/json:
              # JSON的Schema
              schema: 
                # 数组类型
                type: array
                items: 
                  type: string
    # 端点/users的POST方法
    post:
      summary: Creates a user.
      # 请求体
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              properties:
                username:
                  type: string
      responses: 
        '201':
          description: Created
  # 端点/user/{userId}的POST方法
  # {} 中的是路径变量
  /user/{userId}:
    get:
      summary: Returns a user by ID.
      # 参数说明
      parameters:
        - name: userId
          # 这是一个路径变量
          in: path
          required: true
          description: The ID of the user to return.
          # Schema中可以包含字段的验证规则
          schema:
            type: integer
            format: int64
            minimum: 1
      responses:
        '200':
          description: A user object.
          content:
            application/json:
              schema:
                # 对象类型
                type: object
                # 包含属性声明
                properties:
                  id:
                    type: integer
                    format: int64
                    example: 4
                  name:
                    type: string
                    example: Jessica Smith
        # 异常处理
        '400':
          description: The specified user ID is invalid (not a number).
        '404':
          description: A user with the specified ID was not found.
        default:
          description: Unexpected error

components:
  # 定义了一个名为User的Schema
  schemas:
    User:
      properties:
        id:
          type: integer
        name:
          type: string
      # User包含两个属性，都是必须属性
      required:  
        - id
        - name

下面的API引用上述Schema：

paths:
  /users/{userId}:
    get:
      summary: Returns a user by ID.
      parameters:
        - in: path
          name: userId
          required: true
          type: integer
      responses:
        '200':
          description: OK
          content:
            application/json:
              schema:
                # 引用Schema
                $ref: '#/components/schemas/User'

OAS中的所有API端点，相对于Base URL，假设Base URL为https://api.example.com/v1则端点/users的访问路径为https://api.example.com/v1/users。

Open API 3.0允许在servers数组中声明多个Base URL。

API服务器的URL中，可以包含变量：

- url: https://{customerId}.saas-app.com:{port}/v2
    variables:
      customerId:
        default: demo
        description: Customer ID assigned by the service provider
      port:
        enum:
          - '443'
          - '8443'
        default: '443'

你可以在路径级别，甚至操作（方法） 级别，覆盖servers数组：

/ping:
    get:
      servers:
        - url: https://echo.example.com
          description: Override base path for the GET /ping operation

servers数组中的URL可以是相对URL，这种情况下，URL相对于提供OpenAPI定义的Web服务器。

举例来说，如果Open API定义存放在http://localhost:3001/openapi.yaml，则servers元素/v2解析为http://localhost:3001/v2 

请求或者响应的媒体类型，在content元素下声明：

# 200响应，媒体类型为JSON
paths:
  /employees:
    get:
      summary: Returns a list of employees.
      responses:
        '200':      # Response
          description: OK
          content:  # Response body
            application/json:  # Media type
              schema:          # Must-have
                type: object   # Data type

可以指定多种媒体类型：

paths:
  /employees:
    get:
      responses:
        '200':
          description: OK
          content:
            # JSON
            application/json:
             schema: 
               $ref: '#/components/schemas/Employee'
            # XML
            application/xml:
             schema: 
               $ref: '#/components/schemas/Employee'

如果需要为多种媒体类型指定相同的Schema，可以使用

*/*

application/*

paths:
  /info/logo:
    get:
      responses:
        '200':
          description: OK
          content:
            image/*:
             schema: 
               type: string
               format: binary

在OAS中，路径即端点（资源），操作即HTTP方法。

路径可以包括路径变量，例如：

/users/{id}
/organizations/{orgId}/members/{memberId}
/report.{format}

路径的description支持多行文本，还允许使用Markdown语法。

OpenAPI 3.0支持的HTTP方法包括：get, post, put, patch, delete, head, options,  trace。每个路径可以支持多个操作。

OpenAPI 3.0支持通过路径、查询字符串、请求头、Cookie传递参数。对于POST/PUT/PATCH请求，还可以通过请求体传递数据。

这种参数不得定义在路径中，下面是错误的用法：

paths:
  /users?role={role}:

下面则是正确的用法：

paths:
  /users:
    get:
      parameters:
        - in: query
          name: role
          schema:
            type: string
            enum: [user, poweruser, admin]
          required: true

你可以为操作指定一个标识符：

/users:
  get:
    operationId: getUsers

此标识符必须在OAS中是唯一的。 operationId的使用场景包括：

1. 某些代码生成器使用operationId生成对应的方法名

参数需要定义在operation或path的parameters字段下。每个参数包括名字、位置、数据类型（通过schema或content定义）等属性。

参数可以通过以下HTTP元素传递：

1. 路径参数
2. 查询参数
3. 请求头参数
4. Cookie参数 

/users/{id}:
    get:
      parameters:
        - in: path
          # 必须和路径变量{}中的一样
          name: id   
          required: true
          schema:
            type: integer
            minimum: 1
          description: The user ID

路径参数可以是数组，或者对象。这种参数在URL中的串行化形式可以是：

1. 路径风格展开，分号分隔，示例：

   /map/point;x=50;y=20

2. 标签展开，点号前缀，示例：

   /color.R=100.G=200.B=150

3. 简单形式，逗号分隔，示例：

   /users/12,34,56

具体使用哪种串行化风格，通过style、explode关键字指定。

这是最常见的参数形式，出现在URL的?后面。

parameters:
        - in: query
          name: offset
          schema:
            type: integer
        - in: query
          name: limit
          schema:
            type: integer

RFC 3986规定

:/?#[]@!$&'()*+,;=

paths:
  /ping:
    get:
      summary: Checks if the server is alive
      parameters:
        - in: header
          name: X-Request-ID
          schema:
            type: string
            format: uuid
          required: true

parameters:
        - in: cookie
          name: debug
          schema:
            type: integer
            enum: [0, 1]
            default: 0
        - in: cookie
          name: csrftoken
          schema:
            type: string

使用default关键字可以为optional参数提供默认值：

parameters:
  - in: query
    name: offset
    schema:
      type: integer
      minimum: 0
      default: 0
    required: false 

要描述复杂参数的内容，可以使用schema或者content。这两个关键字是互斥的，用于不同的场景下。

大部分情况下，可以考虑使用schema，使用schema可以描述原始类型、串行化为字符串的对象、简单数组。对象、数组的串行化方法在style、explode关键字中定义：

- in: query
  name: color
  schema:
    type: array
    items:
      type: string
  # 串行化为 color=blue,black,brown 形式
  style: form
  explode: false

style、explode无法满足的复杂串行化需求，可以使用content：

parameters:
  - in: query
    name: filter
    content:
      # 细粒度的控制如何串行化为JSON
      application/json:
        schema:
          type: object
          properties:
            type:
              type: string
            color:
              type: string

可以为某个参数指定可选值的集合，这些值的类型必须和参数类型匹配：

parameters:
  - in: query
    name: status
    schema:
      type: string
      enum:
        - available
        - pending
        - sold

常量参数 —— 仅仅包含一个枚举值的参数：

parameters:
  - in: query
    name: rel_date
    required: true
    schema:
      type: string
      enum:
        - now

OpenAPI 3.0允许仅仅有名字，而没有值的参数，在URL中表现为

/path?parm

parameters:
  - in: query
    name: metadata
    schema:
      type: boolean
    allowEmptyValue: true

你也可以在schema中声明nullable属性：

schema:
  type: integer
  format: int32
  nullable: true

可以将一个参数标记为已经废弃：

- in: query
  name: format
  required: true
  schema:
    type: string
    enum: [json, xml, yaml]
  deprecated: true

所谓公共参数，即一个path下，所有方法都使用的参数定义：

/user/{id}:
  parameters:
    - in: path
      name: id
      schema:
        type: integer
      required: true
      description: The user ID
  get:
    ...
  patch:
    ...
  delete:

OpenAPI支持以多种身份验证方式保护API：

1. 基于Authorization头的身份认证：
   1. 不记名令牌（Bearer）
   2. 基本认证
2. 请求头、Cookie、查询字符串中的API Key
3. OAuth2
4. OpenID连接发现

securitySchemes:

  BasicAuth:
    type: http
    scheme: basic

  BearerAuth:
    type: http
    scheme: bearer

  ApiKeyAuth:
    type: apiKey
    in: header
    name: X-API-Key

  OpenID:
    type: openIdConnect
    openIdConnectUrl: https://example.com/.well-known/openid-configuration

  OAuth2:
    type: oauth2
    flows:
      authorizationCode:
        authorizationUrl: https://example.com/oauth/authorize
        tokenUrl: https://example.com/oauth/token
        scopes:
          read: Grants read access
          write: Grants write access
          admin: Grants access to admin operations

为方法添加security字段：

security:
  - ApiKeyAuth: []
  - OAuth2:
      - read
      - write 

引用Schema：

schema: 
  $ref: '#/components/schemas/User'

本地引用：

#/components/schemas/user

远程引用：

1. 当前服务器下其它文件：

   $ref: 'document.json'

2. 文件中的元素：

   $ref: 'document.json#/myElement'

3. 其它服务器中的元素：

   $ref:  http://path/to/your/resource.json#myElement

OpenAPITools/openapi-generator项目，用于从OpenAPI Spec（2和3版本）自动生成客户端库、服务器代码存根、文档以及配置文件。

openapi-generator广泛的支持各种常用语言和框架，对于Go来说，支持的Web框架包括Gin、Echo等。

用于生成OpenAPI 3.0的Go样板代码，以Echo为默认的Web框架。该库致力于尽量的简单化，而非通用化，它不会为所有OpenAPI Schemas生成强类型的Go代码。

默认情况下oapi-codegen会生成包括客户端、服务器、类型定义、内嵌Swagger Spec在内的所有代码。对应命令行选项

-generate=types,client,server,spec

下面我们看看从宠物商店的OpenAPI Spec生成Go样板代码：

openapi: "3.0.0"
info:
  version: 1.0.0
  title: Swagger Petstore
  description: A sample API that uses a petstore as an example to demonstrate features in the OpenAPI 3.0 specification
  termsOfService: http://swagger.io/terms/
  contact:
    name: Swagger API Team
    email: apiteam@swagger.io
    url: http://swagger.io
  license:
    name: Apache 2.0
    url: https://www.apache.org/licenses/LICENSE-2.0.html
servers:
  - url: http://petstore.swagger.io/api
paths:
  /pets:
    get:
      description: |
        Returns all pets from the system that the user has access to
        Nam sed condimentum est. Maecenas tempor sagittis sapien, nec rhoncus sem sagittis sit amet. Aenean at gravida augue, ac iaculis sem. Curabitur odio lorem, ornare eget elementum nec, cursus id lectus. Duis mi turpis, pulvinar ac eros ac, tincidunt varius justo. In hac habitasse platea dictumst. Integer at adipiscing ante, a sagittis ligula. Aenean pharetra tempor ante molestie imperdiet. Vivamus id aliquam diam. Cras quis velit non tortor eleifend sagittis. Praesent at enim pharetra urna volutpat venenatis eget eget mauris. In eleifend fermentum facilisis. Praesent enim enim, gravida ac sodales sed, placerat id erat. Suspendisse lacus dolor, consectetur non augue vel, vehicula interdum libero. Morbi euismod sagittis libero sed lacinia.
        Sed tempus felis lobortis leo pulvinar rutrum. Nam mattis velit nisl, eu condimentum ligula luctus nec. Phasellus semper velit eget aliquet faucibus. In a mattis elit. Phasellus vel urna viverra, condimentum lorem id, rhoncus nibh. Ut pellentesque posuere elementum. Sed a varius odio. Morbi rhoncus ligula libero, vel eleifend nunc tristique vitae. Fusce et sem dui. Aenean nec scelerisque tortor. Fusce malesuada accumsan magna vel tempus. Quisque mollis felis eu dolor tristique, sit amet auctor felis gravida. Sed libero lorem, molestie sed nisl in, accumsan tempor nisi. Fusce sollicitudin massa ut lacinia mattis. Sed vel eleifend lorem. Pellentesque vitae felis pretium, pulvinar elit eu, euismod sapien.
      operationId: findPets
      parameters:
        - name: tags
          in: query
          description: tags to filter by
          required: false
          style: form
          schema:
            type: array
            items:
              type: string
        - name: limit
          in: query
          description: maximum number of results to return
          required: false
          schema:
            type: integer
            format: int32
      responses:
        '200':
          description: pet response
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/Pet'
        default:
          description: unexpected error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
    post:
      description: Creates a new pet in the store. Duplicates are allowed
      operationId: addPet
      requestBody:
        description: Pet to add to the store
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/NewPet'
      responses:
        '200':
          description: pet response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Pet'
        default:
          description: unexpected error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
  /pets/{id}:
    get:
      description: Returns a user based on a single ID, if the user does not have access to the pet
      operationId: find pet by id
      parameters:
        - name: id
          in: path
          description: ID of pet to fetch
          required: true
          schema:
            type: integer
            format: int64
      responses:
        '200':
          description: pet response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Pet'
        default:
          description: unexpected error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
    delete:
      description: deletes a single pet based on the ID supplied
      operationId: deletePet
      parameters:
        - name: id
          in: path
          description: ID of pet to delete
          required: true
          schema:
            type: integer
            format: int64
      responses:
        '204':
          description: pet deleted
        default:
          description: unexpected error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
components:
  schemas:
    Pet:
      allOf:
        - $ref: '#/components/schemas/NewPet'
        - type: object
          required:
          - id
          properties:
            id:
              type: integer
              format: int64

    NewPet:
      type: object
      required:
        - name  
      properties:
        name:
          type: string
        tag:
          type: string    

    Error:
      type: object
      required:
        - code
        - message
      properties:
        code:
          type: integer
          format: int32
        message:
          type: string

通过下面的命令生成样板代码：

go get github.com/deepmap/oapi-codegen/cmd/oapi-codegen
oapi-codegen petstore-expanded.yaml  > petstore.gen.go

 OpenAPI的

/components/schemas

// Type definition for component schema "Error"
type Error struct {
    Code    int32  `json:"code"`
    Message string `json:"message"`
}

// Type definition for component schema "NewPet"
type NewPet struct {
    Name string  `json:"name"`
    Tag  *string `json:"tag,omitempty"`
}

// Type definition for component schema "Pet"
type Pet struct {
    // Embedded struct due to allOf(#/components/schemas/NewPet)
    NewPet
    // Embedded fields due to inline allOf schema
    Id int64 `json:"id"`
}

// Type definition for component schema "Pets"
type Pets []Pet

对于在handler中定义的inline类型，只会生成内联的、匿名的Go结构。这会导致重复的代码，应该避免。

对于paths中的每一个元素，都会生成一个Go Handler函数：

type ServerInterface interface {
    //  (GET /pets)
    FindPets(ctx echo.Context, params FindPetsParams) error
    //  (POST /pets)
    AddPet(ctx echo.Context) error
    //  (DELETE /pets/{id})
    DeletePet(ctx echo.Context, id int64) error
    //  (GET /pets/{id})
    FindPetById(ctx echo.Context, id int64) error
}

请求参数通过以下方式传递：

1. 大部分情况下，函数被编解码到echo.Context中
2. 路径变量作为Handler函数的参数
3. 其它请求头参数、查询参数、Cookie参数被存放在params变量中，例如：
   

   // Parameters object for FindPets
   type FindPetsParams struct {
      Tags  *[]string `json:"tags,omitempty"`
      Limit *int32   `json:"limit,omitempty"`  // 可选参数，作为指针
   }

使用命令行选项

-generate server

func RegisterHandlers(router codegen.EchoRouter, si ServerInterface) {
    wrapper := ServerInterfaceWrapper{
        Handler: si,
    }
    router.GET("/pets", wrapper.FindPets)
    router.POST("/pets", wrapper.AddPet)
    router.DELETE("/pets/:id", wrapper.DeletePet)
    router.GET("/pets/:id", wrapper.FindPetById)
}

使用下面的代码注册Handlers到Echo服务器：

func SetupHandler() {
    var myApi PetStoreImpl  // 这是你的服务器端实现
    e := echo.New()
    petstore.RegisterHandlers(e, &myApi)
    ...
}

类似的，使用命令行选项

-generate chi-server

OpenAPI默认隐含additionalProperties=true，也就是说请求中提供的任何没有明确定义的字段都应该被接受。由于在Go语言中处理这种动态属性需要大量样板代码，oapi-codegen默认假设additionalProperties=false，要改变此默认行为你需要修改OpenAPI Schema：

NewPet:
      required:
        - name
      properties:
        name:
          type: string
        tag:
          type: string
      additionalProperties:
        type: string

这样会生成如下结构：

// NewPet defines model for NewPet.
type NewPet struct {
	Name                 string            `json:"name"`
	Tag                  *string           `json:"tag,omitempty"`
	AdditionalProperties map[string]string `json:"-"`
}

使用命令行选项

-generate=client

// The interface specification for the client above.
type ClientInterface interface {

	// FindPets request
	FindPets(ctx context.Context, params *FindPetsParams, reqEditors ...RequestEditorFn) (*http.Response, error)

	// AddPet request with JSON body
	AddPet(ctx context.Context, body NewPet, reqEditors ...RequestEditorFn) (*http.Response, error)

	// DeletePet request
	DeletePet(ctx context.Context, id int64, reqEditors ...RequestEditorFn) (*http.Response, error)

	// FindPetById request
	FindPetById(ctx context.Context, id int64, reqEditors ...RequestEditorFn) (*http.Response, error)
}


// Client which conforms to the OpenAPI3 specification for this service.
type Client struct {
    // The endpoint of the server conforming to this interface, with scheme,
    // https://api.deepmap.com for example.
    Server string

    // HTTP client with any customized settings, such as certificate chains.
    Client http.Client

    // A callback for modifying requests which are generated before sending over
    // the network.
    RequestEditors []func(ctx context.Context, req *http.Request) error
}

每个OpenAPI Schema中定义的操作都对应了一个客户端函数。 

此工具专门用于从K8S模型类生成Open API模型（以Go结构的形式存放在GetOpenAPIDefinitions函数中）。

下载并安装：

git clone https://github.com/kubernetes/kube-openapi.git

export GOPROXY=https://goproxy.io
export GO111MODULE=on
go install cmd/openapi-gen/openapi-gen.go

要为指定的包生成模型，执行命令：

# 输入包的导入路径
openapi-gen -i git.pacloud.io/pks/helm-operator/pkg/apis/pks/v1 
            # 输出包的导入路径
            -p git.pacloud.io/pks/helm-operator/pkg/apis/pks/v1 
            # 生成的函数所在文件的名称前缀
            -O zz_generated.openapi

你的CRD通常会引用K8S核心库中的模型，因此需要同时为它们生成模型：

openapi-gen -i k8s.io/api/core/v1 -p git.pacloud.io/pks/helm-operator/pkg/apis/core/v1
openapi-gen -i k8s.io/apimachinery/pkg/apis/meta/v1 -p git.pacloud.io/pks/helm-operator/pkg/apis/meta/v1

使用Operator Framework开发CRD的控制器时，你可以使用注释

+k8s:openapi-gen=true

下面的代码调用GetOpenAPIDefinitions函数，生成Swagger 2.0.0 API定义的JSON：

package main

import (
	"encoding/json"
	"fmt"
	pksv1 "git.pacloud.io/pks/helm-operator/pkg/apis/pks/v1"
	"github.com/go-openapi/spec"
	"k8s.io/kube-openapi/pkg/common"
	"log"
	"os"
	"strings"
)

func main() {
	if len(os.Args) <= 1 {
		log.Fatal("version required")
	}
	version := os.Args[1]
	if !strings.HasPrefix(version, "v") {
		version = "v" + version
	}
	oAPIDefs := pksv1.GetOpenAPIDefinitions(func(name string) spec.Ref {
		return spec.MustCreateRef("#/definitions/" + common.EscapeJsonPointer(swaggify(name)))
	})
	defs := spec.Definitions{}
	for defName, val := range oAPIDefs {
		defs[swaggify(defName)] = val.Schema
	}

	swagger := spec.Swagger{
		SwaggerProps: spec.SwaggerProps{
			Swagger:     "2.0",
			Definitions: defs,
			Paths:       &spec.Paths{Paths: map[string]spec.PathItem{}},
			Info: &spec.Info{
				InfoProps: spec.InfoProps{
					Title:   "Helm Release",
					Version: version,
				},
			},
		},
	}
	jsonBytes, err := json.MarshalIndent(swagger, "", "  ")
	if err != nil {
		log.Fatal(err.Error())
	}
	fmt.Println(string(jsonBytes))
}

func swaggify(name string) string {
	name = strings.Replace(name, "git.pacloud.io/pks/helm-operator/pkg/apis", "yun.gmem.cc", -1)
	parts := strings.Split(name, "/")
	hostParts := strings.Split(parts[0], ".")
	for i, j := 0, len(hostParts)-1; i < j; i, j = i+1, j-1 {
		hostParts[i], hostParts[j] = hostParts[j], hostParts[i]
	}
	parts[0] = strings.Join(hostParts, ".")
	return strings.Join(parts, ".")
}

生成的Swagger API，仅仅包含模型信息，也就是definitions字段，但是不包含API端点（paths）。

如果需要生成API端点，则需要启动一个API Server并将需要生成API端点的模型注册进去：

// k8s.io/*包的API非常不稳定，本样例基于kubernetes 1.13.9
package main

import (
	"context"
	"encoding/json"
	"flag"
	"fmt"
	pkscorev1 "git.pacloud.io/pks/helm-operator/pkg/apis/core/v1"
	pksmetav1 "git.pacloud.io/pks/helm-operator/pkg/apis/meta/v1"
	pksv1 "git.pacloud.io/pks/helm-operator/pkg/apis/pks/v1"
	"github.com/go-openapi/spec"
	metainternalversion "k8s.io/apimachinery/pkg/apis/meta/internalversion"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/apimachinery/pkg/runtime"
	"k8s.io/apimachinery/pkg/runtime/schema"
	"k8s.io/apimachinery/pkg/runtime/serializer"
	"k8s.io/apimachinery/pkg/watch"
	openapinamer "k8s.io/apiserver/pkg/endpoints/openapi"
	"k8s.io/apiserver/pkg/registry/rest"
	genericapiserver "k8s.io/apiserver/pkg/server"
	genericoptions "k8s.io/apiserver/pkg/server/options"
	"k8s.io/klog"
	"k8s.io/kube-openapi/pkg/builder"
	"k8s.io/kube-openapi/pkg/common"
	"net"
	"os"
)

// 这个Storage是资源CRUD操作的提供者
type StandardStorage struct {
	cfg ResourceInfo
}

// 强制它实现以下接口
var _ rest.GroupVersionKindProvider = &StandardStorage{}
var _ rest.Scoper = &StandardStorage{}
var _ rest.StandardStorage = &StandardStorage{}

// GroupVersionKindProvider
func (r *StandardStorage) GroupVersionKind(containingGV schema.GroupVersion) schema.GroupVersionKind {
	return r.cfg.gvk
}

// Scoper
func (r *StandardStorage) NamespaceScoped() bool {
	return r.cfg.namespaceScoped
}

// Getter
func (r *StandardStorage) New() runtime.Object {
	return r.cfg.obj
}

func (r *StandardStorage) Create(ctx context.Context, obj runtime.Object, createValidation rest.ValidateObjectFunc, options *metav1.CreateOptions) (runtime.Object, error) {
	return r.New(), nil
}

func (r *StandardStorage) Get(ctx context.Context, name string, options *metav1.GetOptions) (runtime.Object, error) {
	return r.New(), nil
}

// Lister
func (r *StandardStorage) NewList() runtime.Object {
	return r.cfg.list
}

func (r *StandardStorage) List(ctx context.Context, options *metainternalversion.ListOptions) (runtime.Object, error) {
	return r.NewList(), nil
}

// CreaterUpdater
func (r *StandardStorage) Update(ctx context.Context, name string, objInfo rest.UpdatedObjectInfo, createValidation rest.ValidateObjectFunc, updateValidation rest.ValidateObjectUpdateFunc, forceAllowCreate bool, options *metav1.UpdateOptions) (runtime.Object, bool, error) {
	return r.New(), true, nil
}

// GracefulDeleter
func (r *StandardStorage) Delete(ctx context.Context, name string, options *metav1.DeleteOptions) (runtime.Object, bool, error) {
	return r.New(), true, nil
}

// CollectionDeleter
func (r *StandardStorage) DeleteCollection(ctx context.Context, options *metav1.DeleteOptions, listOptions *metainternalversion.ListOptions) (runtime.Object, error) {
	return r.NewList(), nil
}

// Watcher
func (r *StandardStorage) Watch(ctx context.Context, options *metainternalversion.ListOptions) (watch.Interface, error) {
	return nil, nil
}

type ResourceInfo struct {
	gvk             schema.GroupVersionKind
	obj             runtime.Object
	list            runtime.Object
	namespaceScoped bool
}

type TypeInfo struct {
	GroupVersion    schema.GroupVersion
	Resource        string
	Kind            string
	NamespaceScoped bool
}
type Config struct {
	Scheme             *runtime.Scheme
	Codecs             serializer.CodecFactory
	Info               spec.InfoProps
	OpenAPIDefinitions []common.GetOpenAPIDefinitions
	Resources          []TypeInfo
}

func (c *Config) GetOpenAPIDefinitions(ref common.ReferenceCallback) map[string]common.OpenAPIDefinition {
	out := map[string]common.OpenAPIDefinition{}
	for _, def := range c.OpenAPIDefinitions {
		for k, v := range def(ref) {
			out[k] = v
		}
	}
	return out
}
func RenderOpenAPISpec(cfg Config) (string, error) {

	// 总是需要向Scheme注册corev1、metav1
	metav1.AddToGroupVersion(cfg.Scheme, schema.GroupVersion{Version: "v1"})
	unversioned := schema.GroupVersion{Group: "", Version: "v1"}
	cfg.Scheme.AddUnversionedTypes(unversioned,
		&metav1.Status{},
		&metav1.APIVersions{},
		&metav1.APIGroupList{},
		&metav1.APIGroup{},
		&metav1.APIResourceList{},
	)

	// API Server选项
	options := genericoptions.NewRecommendedOptions("/registry/pks.yun.gmem.cc", cfg.Codecs.LegacyCodec(), &genericoptions.ProcessInfo{})
	options.SecureServing.BindPort = 6445
	options.Etcd = nil
	options.Authentication = nil
	options.Authorization = nil
	options.CoreAPI = nil
	options.Admission = nil
	// 自动生成的服务器证书的存放目录
	options.SecureServing.ServerCert.CertDirectory = "/tmp/helm-operator"

	// 启动的API Server，监听的地址
	publicAddr := "localhost"
	ips := []net.IP{net.ParseIP("127.0.0.1")}

	// 尝试自动生成服务器证书
	if err := options.SecureServing.MaybeDefaultWithSelfSignedCerts(publicAddr, nil, ips); err != nil {
		klog.Fatal(err)
	}

	// API Server配置
	serverConfig := genericapiserver.NewRecommendedConfig(cfg.Codecs)
	if err := options.ApplyTo(serverConfig, cfg.Scheme); err != nil {
		return "", err
	}
	serverConfig.OpenAPIConfig = genericapiserver.DefaultOpenAPIConfig(cfg.GetOpenAPIDefinitions, openapinamer.NewDefinitionNamer(cfg.Scheme))
	serverConfig.OpenAPIConfig.Info.InfoProps = cfg.Info
	//                             完成配置，  创建服务器
	genericServer, err := serverConfig.Complete().New("helm-operator-server", genericapiserver.NewEmptyDelegate())
	if err != nil {
		return "", err
	}

	// 这里处理本服务器需要Serve的资源列表
	table := map[schema.GroupVersion]map[string]rest.Storage{}
	{
		for _, ti := range cfg.Resources {
			// 对于每一种资源，根据其GV寻找存储库
			var resmap map[string]rest.Storage
			if m, found := table[ti.GroupVersion]; found {
				resmap = m
			} else {
				// 如果找不到，则创建存储库（每个GV一个存储库）
				resmap = map[string]rest.Storage{}
				table[ti.GroupVersion] = resmap
			}

			gvk := ti.GroupVersion.WithKind(ti.Kind)
			// 创建这种资源的一个对象
			obj, err := cfg.Scheme.New(gvk)
			if err != nil {
				return "", err
			}
			// 创建这种资源的列表对象
			list, err := cfg.Scheme.New(ti.GroupVersion.WithKind(ti.Kind + "List"))
			if err != nil {
				return "", err
			}

			// 为资源创建存储，并Put到它的GV的存储库中
			resmap[ti.Resource] = &StandardStorage{ResourceInfo{
				// GVK信息
				gvk: gvk,
				// 资源和资源列表的原型
				obj:  obj,
				list: list,
				// 提示此资源是否命名空间化
				namespaceScoped: ti.NamespaceScoped,
			}}
		}
	}
	for gv, resmap := range table {
		// 为每个组创建API组信息
		apiGroupInfo := genericapiserver.NewDefaultAPIGroupInfo(gv.Group, cfg.Scheme, metav1.ParameterCodec, cfg.Codecs)
		storage := map[string]rest.Storage{}
		for r, s := range resmap {
			storage[r] = s
		}
		apiGroupInfo.VersionedResourcesStorageMap[gv.Version] = storage

		// 并安装到此服务器
		if err := genericServer.InstallAPIGroup(&apiGroupInfo); err != nil {
			return "", err
		}
	}
	// 构件API规范，也就是Swagger 2.0 Spec
	spec, err := builder.BuildOpenAPISpec(genericServer.Handler.GoRestfulContainer.RegisteredWebServices(), serverConfig.OpenAPIConfig)
	if err != nil {
		return "", err
	}
	data, err := json.MarshalIndent(spec, "", "  ")
	if err != nil {
		return "", err
	}
	return string(data), nil
}

func main() {
	flag.Parse()
	if len(os.Args) <= 1 {
		panic("version required")
	}
	version := os.Args[1]

	scheme := runtime.NewScheme()
	// 将我们需要处理的类型加入到scheme
	scheme.AddKnownTypeWithName(schema.GroupVersion{Group: "pks.yun.gmem.cc", Version: "v1"}.WithKind("Release"), &pksv1.Release{})
	scheme.AddKnownTypeWithName(schema.GroupVersion{Group: "pks.yun.gmem.cc", Version: "v1"}.WithKind("ReleaseList"), &pksv1.Release{})
	scheme.AddKnownTypeWithName(schema.GroupVersion{Group: "pks.yun.gmem.cc", Version: "v1"}.WithKind("WatchEvent"), &pksv1.Release{})

	spec, err := RenderOpenAPISpec(Config{
		Info: spec.InfoProps{
			Version: version,
			Title:   "Helm Operator OpenAPI",
		},
		Scheme: scheme,
		Codecs: serializer.NewCodecFactory(scheme),
		OpenAPIDefinitions: []common.GetOpenAPIDefinitions{
			pkscorev1.GetOpenAPIDefinitions,
			pksmetav1.GetOpenAPIDefinitions,
			pksv1.GetOpenAPIDefinitions,
		},
		Resources: []TypeInfo{
			{
				GroupVersion:    schema.GroupVersion{Group: "pks.yun.gmem.cc", Version: "v1"},
				Kind:            "Release",
				Resource:        "Release",
				NamespaceScoped: true,
			},
		},
	})
	if err != nil {
		klog.Fatal(err.Error())
	}
	fmt.Println(spec)
}

可以使用下面的插件：

plugins {
  id 'org.hidetake.swagger.generator' version '2.18.1'
}

根据API规格的版本，选择依赖：

dependencies {
  swaggerCodegen 'io.swagger:swagger-codegen-cli:2.4.2'             // Swagger Codegen V2
  swaggerCodegen 'io.swagger.codegen.v3:swagger-codegen-cli:3.0.5'  // Swagger Codegen V3
  swaggerCodegen 'org.openapitools:openapi-generator-cli:3.3.4'     // penAPI Generator
}

控制代码生成的配置片段：

swaggerSources {
  petstore {
    inputFile = file('petstore.yaml')
    code {
      language = 'spring'
      configFile = file('config.json')
    }
  }
}

执行命令：

./gradlew generateSwaggerCode

build/swagger-code-petstore

configFile字段可以指定一个外部（本质上就是

swagger-codegen-cli generate

{
  "library": "spring-mvc",
  "modelPackage": "example.model",
  "apiPackage": "example.api",
  "invokerPackage": "example"
}

对于language=java，常用配置项列表：

The post OpenAPI学习笔记 appeared first on 绿色记忆.

Maven本身只是一套框架，它的功能基于全部依赖于插件来实现。因此，掌握插件开发深度定制Maven的必修课。

插件本身也是Maven构件，构件标识的命名约定是

<yourplugin>-maven-plugin

maven-<yourplugin>-plugin

org.apache.maven.plugins

每个Maven插件包含一或多个Mojo，每个Mojo实现了一个插件目标（goal），Mojo通常编写为Java类。插件就是一系列Mojo的集合。

插件必须在自己的JAR包的

META-INF/maven/plugin.xml

本章我们编写一个最简单的插件，它不需要参数，仅仅简单的打印一段消息。

可以将mojo理解为插件的一个目标（goal）的实现类。你需要使用@Mojo注解来声明一个类是mojo：

package sample.plugin;
 
import org.apache.maven.plugin.AbstractMojo;
import org.apache.maven.plugin.MojoExecutionException;
import org.apache.maven.plugins.annotations.Mojo;
 
@Mojo( name = "sayhi")
public class GreetingMojo extends AbstractMojo
{
    public void execute() throws MojoExecutionException
    {
        getLog().info( "Hello, world." );
    }
}

抽象类AbstractMojo包含了实现mojo所需的基础代码，我们仅仅需要实现execute方法就可以了。execute方法可以抛出两种异常：

1. MojoExecutionException：表示意外错误发生，控制台会显示BUILD ERROR消息
2. MojoFailureException：表示非意外的错误发生，例如编译是白。控制台会显示BUILD FAILURE消息

插件本身也是Maven项目，因此你需要编写POM，示例：

<project>
  <modelVersion>4.0.0</modelVersion>
 
  <groupId>sample.plugin</groupId>
  <artifactId>hello-maven-plugin</artifactId>
  <version>1.0-SNAPSHOT</version>
  <!-- 打包方式必须指定为maven-plugin -->
  <packaging>maven-plugin</packaging>
 
  <name>Sample Parameter-less Maven Plugin</name>
 
  <dependencies>
    <dependency>
      <groupId>org.apache.maven</groupId>
      <artifactId>maven-plugin-api</artifactId>
      <version>3.0</version>
    </dependency>
 
    <!-- 插件注解 -->
    <dependency>
      <groupId>org.apache.maven.plugin-tools</groupId>
      <artifactId>maven-plugin-annotations</artifactId>
      <version>3.4</version>
      <scope>provided</scope>
    </dependency>
  </dependencies>
</project>

另创建一个Maven项目，添加以下配置： 

<build>
  <plugins>
    <plugin>
      <groupId>sample.plugin</groupId>
      <artifactId>hello-maven-plugin</artifactId>
      <version>1.0-SNAPSHOT</version>
    </plugin>
  </plugins>
</build>

然后调用我们的插件：

# mvn groupId:artifactId:version:goal
mvn sample.plugin:hello-maven-plugin:1.0-SNAPSHOT:sayhi

上面调用插件的mvn命令参数非常冗长，缩短的方式有几种：

1. 如果希望执行本地仓库中，插件的最新版本，则version可以省略
2. 可以为插件分配快捷前缀，如果插件命名遵循了

   ${prefix}-maven-plugin

   格式则prefix自动为前缀。调用命令可以简化为

   mvn sample.plugin:hello:sayhi

3. 可以将组标识添加到Maven的settings.xml的pluginGroups，这样就可以进一步省略组标识：
   

   <pluginGroups>
       <pluginGroup>sample.plugin</pluginGroup>
   </pluginGroups>

   调用命令简化为

   mvn hello:sayhi

在目标项目中增加如下配置：

<build>
  <plugins>
    <plugin>
      <groupId>sample.plugin</groupId>
      <artifactId>hello-maven-plugin</artifactId>
      <version>1.0-SNAPSHOT</version>
      <executions>
        <execution>
          <phase>compile</phase>
          <goals>
            <goal>sayhi</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

则在构建生命周期的compile阶段，sayhi目标自动执行。

你可以从原型创建插件项目，避免手写样板代码： 

mvn archetype:generate \
    -DgroupId=sample.plugin \
    -DartifactId=hello-maven-plugin \
    -DarchetypeGroupId=org.apache.maven.archetypes \
    -DarchetypeArtifactId=maven-archetype-plugin

Mojo可以支持参数，目标项目的POM中可以声明传入的参数值，你也可以在命令行中以-D系统属性的风格传入参数值。

要定义Mojo参数，只需要为Mojo类的字段添加注解：

@Parameter( property = "sayhi.greeting", defaultValue = "Hello World!" )
private String greeting;

上面的代码定义了一个参数greeting，并给定了默认值。此参数对应的系统属性为sayhi.greeting。

参数值可以使用

${project.version}

<plugin>
  <groupId>sample.plugin</groupId>
  <artifactId>hello-maven-plugin</artifactId>
  <version>1.0-SNAPSHOT</version>
  <configuration>
    <!-- 这里使用参数的字段名而非系统属性 -->
    <greeting>Welcome</greeting>
  </configuration>
</plugin>

在插件参数表达式中，你可以使用以下对象：

MavenSession

MavenSession.getLocalRepository()

MavenSession.getProjects()

MavenSession.getRepositorySession()

MavenSession.getCurrentProject()

MavenProject.getExecutionProject()

MavenSession.getSettings()

MavenSession.getExecutionRootDirectory()

System.getProperty( "user.dir" )

MojoExecution.getMojoDescriptor().getPluginDescriptor() 

配置中的文本转换为Mojo参数字段的规则如下：

Mojo字段可以使用数组、集合类：

@Parameter
private String[] myArray;

对应POM配置示例： 

<myArray>
    <param>value1</param>
    <param>value2</param>
</myArray>

映射类型的字段POM配置示例，参数名为myMap： 

<myMap>
    <key1>value1</key1>
    <key2>value2</key2>
</myMap>

Properties类型的POM配置示例，参数名为myProperties：

<myProperties>
  <property>
    <name>propertyName1</name>
    <value>propertyValue1</value>
  <property>
  <property>
    <name>propertyName2</name>
    <value>propertyValue2</value>
  <property>
</myProperties>

Mojo参数字段也可以是任意对象类型： 

@Parameter
private MyObject myObject;

对应POM配置示例：  

<myObject>
    <myField>test</myField>
</myObject>

用于测试单个Mojo的功能，并仿冒Maven的其它部分。

你可以像编写普通Junit测试用例那样，进行Mojo单元测试。但是，大部分情况下Mojo都需要被注入一个Maven项目的引用，可以

extends PlexusTestCase

此工具专门用于测试

org.apache.maven.reporting.AbstractMavenReport#execute()

在一个真实的Maven Build中测试Mojo，你需要提供一个测试用的Maven项目，插件需要安装到本地仓库。

Maven验证器为你的JUnit测试用例提供以下功能：

1. 启动Maven
2. 根据构建的输出、日志信息进行断言
3. 从src/test/resources目录抽取出一个Maven项目，并存放到临时工作目录中，针对此项目进行测试
4. 操控本地仓库

Maven项目本身也在用此验证器进行集成测试，下面是一些说明如何使用该验证器的代码片段：

public class TrivialMavenVerifierTest extends TestCase {
    public void testMyPlugin() throws Exception {
        // 从/src/test/resources/中抽取测试用的Maven项目
        File testDir = ResourceExtractor.simpleExtractResources( getClass(), "/my-dummy-maven-project" );
 
        Verifier verifier;
 
        // 首选需要保证，此test项目产生的构件已经从本地仓库移除
        verifier = new Verifier( testDir.getAbsolutePath() );
        verifier.deleteArtifact( "org.apache.maven.its.itsample", "parent", "1.0", "pom" );
        verifier.deleteArtifact( "org.apache.maven.its.itsample", "checkstyle-test", "1.0", "jar" );
        verifier.deleteArtifact( "org.apache.maven.its.itsample", "checkstyle-assembly", "1.0", "jar" );
 
        // 执行Maven命令
        List cliOptions = new ArrayList();
        cliOptions.add( "-N" );
        verifier.executeGoal( "install" );
 
        // 验证上述命令是否成功
        verifier.verifyErrorFreeLog();
 
        // 清理，准备下一个命令的执行
        verifier.resetStreams();
    }
}

这是一个插件，可以调用Maven，并提供一些BeanShell或Groovy的测试脚本： 

<build>
  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-invoker-plugin</artifactId>
      <version>1.10</version>
      <configuration>
        <projectsDirectory>src/it</projectsDirectory>
        <pomIncludes>
          <pomInclude>**/pom.xml</pomInclude>
        </pomIncludes>
        <postBuildHookScript>verify</postBuildHookScript>
      </configuration>
      <executions>
        <execution>
          <goals>
            <goal>run</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
    ...
  </plugins>
</build>

要调试Maven插件，可以使用远程调试的方式。

你需要用

mvnDebug

示例：

mvnDebug cc.gmem.yun.maven.plugins:maven-archetype-plugin:3.1.1:create-from-project \
    -Darchetype.properties=archetype.properties -Darchetype.postPhase=package -Darchetype.partialArchetype=true

# Preparing to execute Maven in debug mode
# Listening for transport dt_socket at address: 8000

配合Intellij IDEA等开发工具，可以很容易实现本地调试。

在你的插件项目中，创建一个Maven的运行配置，设置环境变量：

MAVEN_DEBUG_OPTS="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000"

然后Maven的Command line调用你的插件的Mojo，配置好了点击调试按钮即可。 

import org.apache.maven.execution.MavenSession;
import org.apache.maven.plugin.AbstractMojo;
import org.apache.maven.plugin.MojoExecution;
import org.apache.maven.plugin.descriptor.PluginDescriptor;
import org.apache.maven.plugins.annotations.Component;
import org.apache.maven.plugins.annotations.Execute;
import org.apache.maven.plugins.annotations.InstantiationStrategy;
import org.apache.maven.plugins.annotations.LifecyclePhase;
import org.apache.maven.plugins.annotations.Mojo;
import org.apache.maven.plugins.annotations.Parameter;
import org.apache.maven.plugins.annotations.ResolutionScope;
import org.apache.maven.project.MavenProject;
import org.apache.maven.settings.Settings;

       // 此Mojo对应的目标的名称
@Mojo( name = "<goal-name>",
       aggregator = <false|true>, 
       configurator = "<role hint>",
       // 执行策略
       executionStrategy = "<once-per-session|always>",
       inheritByDefault = <true|false>,
       // 实例化策略
       instantiationStrategy = InstantiationStrategy.<strategy>,
       // 如果用户没有在POM中明确设置此Mojo绑定到的phase，那么绑定一个MojoExecution到那个phase
       defaultPhase = LifecyclePhase.<phase>,
       requiresDependencyResolution = ResolutionScope.<scope>,
       requiresDependencyCollection = ResolutionScope.<scope>,
       // 提示此Mojo需要被直接调用（而非绑定到生命周期阶段）
       requiresDirectInvocation = <false|true>,
       // 提示此Mojo不能在离线模式下运行
       requiresOnline = <false|true>,
       // 提示此Mojo必须在一个Maven项目内运行
       requiresProject = <true|false>,
       // 提示此Mojo是否线程安全，线程安全的Mojo支持在并行构建中被并发的调用
       threadSafe = <false|true> ) // (since Maven 3.0)

// 何时执行此Mojo
@Execute( goal = "<goal-name>",           // 如果提供goal，则隔离执行此Mojo
          phase = LifecyclePhase.<phase>, // 在此生命周期阶段自动执行此Mojo
          lifecycle = "<lifecycle-id>" )  // 在此生命周期中执行此Mojo
public class MyMojo
    extends AbstractMojo
{
    
    @Parameter( name = "parameter",
                // 在POM中可使用别名来配置参数
                alias = "myAlias",
                property = "a.property",
                defaultValue = "an expression, possibly with ${variables}",
                readonly = <false|true>,
                required = <false|true> )
    private String parameter;
 
    @Component( role = MyComponentExtension.class,
                hint = "..." )
    private MyComponent component;
 
 
    @Parameter( defaultValue = "${session}", readonly = true )
    private MavenSession session;
 
    @Parameter( defaultValue = "${project}", readonly = true )
    private MavenProject project;
 
    @Parameter( defaultValue = "${mojoExecution}", readonly = true )
    private MojoExecution mojo;
 
    @Parameter( defaultValue = "${plugin}", readonly = true )
    private PluginDescriptor plugin;
 
    @Parameter( defaultValue = "${settings}", readonly = true )
    private Settings settings;
 
    @Parameter( defaultValue = "${project.basedir}", readonly = true )
    private File basedir;
 
    @Parameter( defaultValue = "${project.build.directory}", readonly = true )
    private File target;
 
    public void execute()
    {
    }
}

The post Maven插件开发 appeared first on 绿色记忆.

原型（Archetype）是Maven的模板工具箱。利用原型，可以很容易的应用组织在软件实现方面的最佳实践，减轻搭建项目框架的负担。

原型机制支持“增量“方式，也就是说，你通过原型生成一个项目后，可以再次用另外一个原型来生成额外的内容到此项目中。这种增量机制允许将项目的不同方面/部分封装到多个原型中，增强可复用、可组合性。

从Maven原型创建项目，需要使用maven-archetype-plugin插件。

一般情况下Maven原型都存放在远程仓库中，Maven中心仓库默认可用，你也可以修改Maven的settings.xml添加别的原型仓库：

<repository>
  <id>archetype</id>
  <url>https://m2.gmem.cc/path/to/repo/</url>
</repository>

<!-- 如果需要身份验证 -->
<server>
  <id>archetype</id>
  <username>user.name</username>
  <password>s3cr3t</password>
</server>

交互式调用archetype:generate时，maven-archetype-plugin会提示你从某个仓库选择原型，输入对应的序号即可。

使用filter可以减少原型列表的长度：

mvn archetype:generate -Dfilter=org.apache:struts 

选择原型后，maven-archetype-plugin会提示你输入一系列属性的值，最终生成新项目。

生成项目时，你需要提供新项目的GAV、package，原型还可能规定了一系列其它必须属性，这些属性会用来：

1. 填充Velocity模板中的变量
2. 如果目录、文件名包含

   __property__

   ，则被替换为对应的属性值

3. __artifactId__

   被替换为项目的ArtifactId
4. 在多模块项目中，

   __rootArtifactId__

   被替换为根项目的ArtifactId

你可以从零开始，开发一个原型，也可以从现有的项目反向生成原型。

原型描述符用于存储原型的元数据，它存放在原型JAR的 META-INF/maven/archetype-metadata.xml中。下面是一个例子：

<archetype-descriptor xmlns="https://maven.apache.org/plugins/maven-archetype-plugin/archetype-descriptor/1.1.0" 
                      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="https://maven.apache.org/plugins/maven-archetype-plugin/archetype-descriptor/1.1.0 
                      http://maven.apache.org/xsd/archetype-descriptor-1.1.0.xsd"
  <!-- 原型的名称-->
          <!-- 是否偏原型 -->
  name=.. partial=.. >
  <!-- 用户必须提供的属性列表 -->
  <requiredProperties>
    <requiredProperty key=.. >
      <defaultValue/>
      <validationRegex/>
    </requiredProperty>
  </requiredProperties>
 
  <!-- 文件集定义 -->
  <fileSets>
    <!-- 文件集说明了在生成项目的过程中，如何使用原型JAR中的文件 -->
    <!-- filtered 此文件集是否可以被过滤，如果true，则意味着这些文件被用作Velocity模板。否则原样拷贝 -->
    <!-- packaged 此文件集是否应该被生成/拷贝到package属性所指定的目录层次中 -->
    <!-- encoding 文件的编码方式 -->
    <fileSet filtered=.. packaged=.. encoding=.. >
      <directory/> <!-- 此文件集位于的目录 -->
      <includes><include></include></includes>  <!-- 包含的文件列表，Ant语法 -->
      <excludes/> <!-- 排除的文件列表，Ant语法 -->
    </fileSet>
  </fileSets>
 
  <!-- 模块定义 -->
  <modules>
    <module id=.. dir=.. name=.. >
      <fileSets>
        <fileSet filtered=.. packaged=.. encoding=.. >
          <directory/>
          <includes/>
          <excludes/>
        </fileSet>
      </fileSets>
      <!-- 模块可以嵌套 -->
      <modules>
        <module>...recursion...<module>
      </modules>
    </module>
  </modules>
</archetype-descriptor> 

下面是原型描述符的最小化示例：

<?xml version="1.0" encoding="UTF-8"?>
<archetype-descriptor name="basic">
  <requiredProperties>
    <requiredProperty key="property-with-default">
      <defaultValue>default-value</defaultValue>
    </requiredProperty>
    <requiredProperty key="property-without-default"/>
  </requiredProperties>
  <fileSets>
    <fileSet filtered="true" packaged="true">
      <directory>src/main/java</directory>
      <includes>
        <include>**/*.java</include>
      </includes>
    </fileSet>
  </fileSets>
</archetype-descriptor>

本节我们演示一下，如何从一个现有的项目，反向生成原型。

首先创建一个空白的Maven项目：

mvn archetype:generate -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.0 \
    -DgroupId=cc.gmem.study.mvn -DartifactId=archetype-proto -DinteractiveMode=false

进入示例项目，生成原型：

cd JavaEE/projects/idea/archetype-study/archetype-proto/
mvn archetype:create-from-project

如果没有错误，生成的原型项目位于target/generated-sources/archetype目录：

.
└── archetype                                         # 原型项目的根目录
    ├── pom.xml                                       # 原型项目的POM
    ├── src
    │   ├── main
    │   │   └── resources
    │   │       ├── archetype-resources               # 原始项目的所有内容，亦即从此原型生成的项目的所有内容
    │   │       │   ├── __artifactId__.iml
    │   │       │   ├── .idea
    │   │       │   └── src
    │   │       │       └── main
    │   │       │           └── java
    │   │       │               └── App.java          # 注意包结构消失了
    │   │       │   └── pom.xml
    │   │       └── META-INF
    │   │           └── maven
    │   │               └── archetype-metadata.xml    # 原型元数据
    │   └── test
    │       └── resources
    │           └── projects
    │               └── basic
    │                   ├── archetype.properties      # 用于测试原型
    │                   └── goal.txt
    └── target

可以看到，上面的例子将原始项目的IDE元数据文件也放入到原型中了，这不是我们期望的，可以使用属性文件排除。

在原始项目中添加属性文件：

excludePatterns=.idea/**

然后重新生成原型：

mvn clean  archetype:create-from-project -Darchetype.properties=archetype.properties

可以看到.idea目录被排除掉了。

从上面的例子还可以看到，生成的原型项目中App.java的包消失了。

实际上消失的是包前缀，它的值等于传递给create-from-project的package属性，如果不传递，则使用原始项目的包前缀。

生成的原型元数据中有如下片段：

<fileSet filtered="true" packaged="true" encoding="UTF-8">
  <directory>src/main/java</directory>
  <includes>
    <include>**/*.java</include>
  </includes>
</fileSet>

其中package=true含义是，从原型生成项目时，archetype-resources/src/main/java中的内容会被包裹在$package指定的目录结构中。

在文件中添加一个配置：

application=App

它的含义是，搜索原始项目的路径、文件中的文本，将App都更换为Application变量。对于本示例项目来说，其效果是：

1. App.java的文件名替换为

   __application__.java

2. App.java中的类名被替换为

   ${application} 

这是一种简单粗暴的查找替换，使用时务必小心。 

仍然使用上个例子，首先，构建并安装原型：

mvn clean  archetype:create-from-project \
    -Darchetype.properties=archetype.properties -Darchetype.postPhase=install

本地原型目录会自动更新。

基于该原型创建一个新项目：

mvn archetype:generate -B -DarchetypeGroupId=cc.gmem.study.mvn \
    -DarchetypeArtifactId=archetype-proto-archetype -DarchetypeVersion=1.0-SNAPSHOT \
    -DgroupId=cc.gmem -DartifactId=project -Dversion=1.0-SNAPSHOT \
                              # 传递自定义属性
    -Dpackage=cc.gmem.project -Dapplication=ProjectApp

检查生成的项目， 可以看到目录结构如下：

.
└── project
    ├── archetype-metadata.yaml
    ├── pom.xml
    └── src
        └── main
            └── java
                └── cc
                    └── gmem
                        └── project
                            ├── ProjectApp.java
                            └── service
                                └── ProjectAppService.java 

要在Maven中使用原型来生成项目骨架，必须依赖于maven-archetype-plugin。此插件依赖Java 7+版本，主流IDE内置了该插件。

maven-archetype-plugin使用Velocity作为模板引擎。

这是Maven原型项目的packaging，这种打包方式的生命周期绑定以下目标

1. archetype:jar，绑定到package阶段，生成原型的JAR包
2. archetype:integration-test，绑定到integration-test阶段
3. archetype:update-local-catalog，绑定到install阶段，更新本地原型目录

从原型生成项目骨架：

1. 如果从原型创建新的Maven项目，则输出到名为项目artifactId的目录
2. 如果使用一个偏原型（partial archetype）更新已有项目，则输出到当前目录

用户需要从原型目录中选择一个原型，从远程仓库将原型拉取下来，然后生成项目。调用示例：

# -B表示批处理模式，等价于-DinteractiveMode=false
mvn archetype:generate -B -DarchetypeGroupId=org.apache.maven.archetypes \
    -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.1 \
    -DgroupId=com.company -DartifactId=project -Dversion=1.0-SNAPSHOT \
    -Dpackage=com.company.project

其中通过-D传递的，是目标的参数。

可用参数包括：

该目标用于从现有的项目（以下称原始项目）反向生成原型。它会读取项目中的源文件、资源文件，你通过.property指定的原型属性文件，并产生一个原型项目。原始项目中的文本文件会被转换为Velocity模板。

爬取文件系统中的Maven仓库，产生一个Maven原型目录文件。

The post Maven原型系统 appeared first on 绿色记忆.

如何在云原生环境下进行CI/CD，我们先前有一些经验：

1. 使用Jenkins + Jenkins的Kubernete插件
2. 在K8S中按需、动态创建执行CI/CD流水线的Agent
3. 开发Jenkins共享库，简化编写流水线的难度
4. 为每套环境（development staging production，除了production允许有多套）创建独立的命名空间，对应K8S的namespace和Jenkins的folder
5. 利用一个特殊的Jenkins Job来复制、创建新的环境

先前我们搭建的云原生平台主要供内部使用，所以不注重产品化。虽然环境可以复制，但是每个Jenkins Job还是依赖于人工通过Jenkins UI创建。

这种人肉管理Job的方式很不方便，现在做的产品化PaaS平台（以下简称平台）也不希望使用Jenkins UI，仅仅想将它作为流水线执行引擎的一种实现。

除了Web UI之外，Jenkins还提供了RESTful风格的API，平台可以通过这些API和Jenkins交互，API的端点路径位于/api。

目前，REST API有三种风格：

1. XML API，载荷格式为XML，支持XPath查询。API端点路径位于https://ci.gmem.cc/api/xml?...
2. JSON API，载荷格式JSON，支持JSONP
3. Python API，载荷格式可以通过

   eval(urllib.urlopen("...").read())

   转换为Python对象

通过REST API，可以创建Jenkins Job、复制Jenkins Job、管理构建队列，以及重启Jenkins服务器。 

直接调用REST API比较麻烦，以下语言已经有了对API的封装，也就是Jenkins 客户端库：

1. Ruby：Jenkins API Client
2. Java：jenkins-rest
   
3. Python：JeninsAPI

使用Jenkins REST API，你的应用程序就可以管理Jenkins服务器，创建Jenkins Job了。但是，社区还有更具K8S风格的解决方案 —— Jenkins Operator，本章先讨论它的依赖job-dsl-plugin。

Jenkins Job DSL / Plugin项目包含两个部分：

1. 一个DSL，允许用户用Groovy语言来定义一个Job
2. 一个Jenkins 插件，根据用户定义的上述DSL，生成Jenkins Job

此插件的目的是，尽可能简化Jenkins Job的创建，它允许用户使用编程的方式来配置Job，并且把所有Job的通用元素隐藏在DSL背后。

举例来说，你有个项目，需要单元测试、构建、集成测试、部署四个任务，那么基于DSL只需要编写：

def gitUrl = 'git://github.com/jenkinsci/job-dsl-plugin.git'

job('PROJ-unit-tests') {
    scm {
        git(gitUrl)
    }
    triggers {
        scm('*/15 * * * *')
    }
    steps {
        maven('-e clean test')
    }
}

job('PROJ-sonar') {
    scm {
        git(gitUrl)
    }
    triggers {
        cron('15 13 * * *')
    }
    steps {
        maven('sonar:sonar')
    }
}

job('PROJ-integration-tests') {
    scm {
        git(gitUrl)
    }
    triggers {
        cron('15 1,13 * * *')
    }
    steps {
        maven('-e clean integration-test')
    }
}

job('PROJ-release') {
    scm {
        git(gitUrl)
    }
    steps {
        maven('-B release:prepare release:perform')
        shell('cleanup.sh')
    }
}

job-dsl-plugin的特性包括：

1. 可以通过DSL直接控制XML，因此任何config.xml可以存在的片段都可以通过DSL操控
2. DSL提供了很多助手方法，方便通用的Job配置，包括scm、trigger、steps
3. DSL可以直接编写在Job里
4. DSL可以放在SCM中，并且通过标准的SCM触发机制拉取

使用job-dsl-plugin的步骤如下：

1. 创建一个 Free-style Project类型的Jenkins Job，用于运行DSL，这种Job叫做Seed Job
2. 配置Seed Job，添加Process Job DSLs类型的Build Step，在其中编写DSL
3. 运行Seed Job来生成新的、实际有用的Job

本节列出job-dsl-plugin的常用DSL。完整的DSL API请参考官方文档。

下面的API用于创建不同类型的Job：

// freeStyleJob的别名
job(String name, Closure closure = null)          

freeStyleJob(String name, Closure closure = null) 

buildFlowJob(String name, Closure closure = null) 

ivyJob(String name, Closure closure = null)       

matrixJob(String name, Closure closure = null)    

mavenJob(String name, Closure closure = null)     

multiJob(String name, Closure closure = null)     

workflowJob(String name, Closure closure = null)  

multibranchWorkflowJob(String name, Closure closure = null)

上述DSL都返回一个Job对象，并且可以后续继续修改Job： 

def myJob = freeStyleJob('SimpleJob')
myJob.with {
    description 'A Simple Job'
}

下面的API用于创建视图：

listView(String name, Closure closure = null)            

sectionedView(String name, Closure closure = null)       

nestedView(String name, Closure closure = null)          

deliveryPipelineView(String name, Closure closure = null)

buildPipelineView(String name, Closure closure = null)   

buildMonitorView(String name, Closure closure = null)    

categorizedJobsView(String name, Closure closure = null)

如果安装（现在的版本默认已经安装）了CloudBees Folders Plugin插件，下面的API可以用于创建目录：

folder(String name, Closure closure = null)

目录内的对象可以用路径的形式创建：

folder('project-a')

freeStyleJob('project-a/compile')

listView('project-a/pipeline')

folder('project-a/testing')

下面的API用于调度一个Job：

queue(String jobName)
queue(Job job)

下面的API可以用于读取工作区中的文件：

InputStream streamFileFromWorkspace(String filePath)
String readFileFromWorkspace(String filePath)
String readFileFromWorkspace(String jobName, String filePath)

任何DSL脚本都可以访问名为out的变量，调用它可以打印日志到构建日志（Console Output）中：

out.println('Hello from a Job DSL script!')

如果希望输出信息到Jenkins日志，可以：

import java.util.logging.Logger

Logger logger = Logger.getLogger('org.example.jobdsl')
logger.info('Hello from a Job DSL script!') 

如果Job DSL不支持某种Option，你可以使用Configure Block来扩展DSL的能力。

job('example') {
    ...
    configure { project ->
        project / buildWrappers / EnvInjectPasswordWrapper {
            injectGlobalPasswords(true)
        }
    }
}

Configure Block允许你对配置文件config.xml（Jenkins的各种对象的配置都放在自己独立目录的config.xml中）进行直接访问。此块的闭包接收到一个groovy.util.Node参数，表示一个XML节点，具体是什么节点取决于Configure Block所在的上下文：

job('example-1') {
    configure { node ->
        // node为project
    }
}

mavenJob('example-2') {
    configure { node ->
        // node为maven2-moduleset
    }
}

listView('example') {
    configure { node ->
        // node为hudson.model.ListView
    }
}

你可以使用groovy.util.Node的API操控Jenkins对象的配置元素，但是代码会很丑陋。DSL提供了特殊的语法以简化XML操作，需要注意：

1. 所有查询返回NodeList，因此像操控第一个元素的话需要调用 nodes[0]
2. 操作符

   +

   用于添加兄弟节点
3. 不能访问不存在的子元素
4. 如果元素名和Groovy关键字、Groovy操作符、任何上层上下文对象的属性（例如properties）冲突，则必须使用引号：
   

   configure {
       //   和上下文对象属性冲突  带有点号
       it / 'properties' / 'com.example.Test' {
           'switch'('on')
       }
   }

5. 为了简化操作，两个操作符被重写：

   1. /

      根据名称查找子节点，如果子节点不存在，则会创建。可以指定子节点属性以匹配第一个带有属性的子节点

   2. <<

      添加为子节点，右操作数可以是字符串，为子节点名称；也可以是closure，其行为类似于NodeBuilder

下面是一些例子：

// 简单的例子
configure {
    // it是groovy.util.Node对象，表示Job的config.xml的根元素project
    def aNode = it
    // anotherNode是project的子元素
    def anotherNode = aNode / 'blockBuildWhenDownstreamBuilding'
    // 设置此元素的值
    anotherNode.setValue('true')

    // 链式调用，注意 / 的操作符优先级很低
    (it / 'blockBuildWhenUpstreamBuilding').setValue('true')
}


// 添加权限
job('example') {
    configure { project ->
        def matrix = project / 'properties' / 'hudson.security.AuthorizationMatrixProperty' {
            permission('hudson.model.Item.Configure:jill')
            permission('hudson.model.Item.Configure:jack')
        }
        matrix.appendNode('permission', 'hudson.model.Run.Delete:jryan')
    }
}
/*
对应XML：
<project>
    <properties>
        <hudson.security.AuthorizationMatrixProperty>
            <permission>hudson.model.Item.Configure:jill</permission>
            <permission>hudson.model.Item.Configure:jack</permission>
            <permission>hudson.model.Run.Delete:jryan</permission>
        </hudson.security.AuthorizationMatrixProperty>
    </properties>
</project>
对应DSL：
job('example') {
    authorization {
        permission(Permissions.ItemConfigure, 'jill')
        permission(Permissions.ItemConfigure, 'jack')
        permission(Permissions.RunDelete, 'jryan')
    }
}
*/


// 配置日志轮换插件
job('example') {
    configure { project ->
        // Doesn't take into account existing node
        project << logRotator {
            daysToKeep(-1)
            numToKeep(10)
            artifactDaysToKeep(-1)
            artifactNumToKeep(-1)
        }

        // Alters existing value
        (project / logRotator / daysToKeep).value = 2
    }
}
/*
对应XML：
<project>
    <logRotator>
        <daysToKeep>2</daysToKeep>
        <numToKeep>10</numToKeep>
        <artifactDaysToKeep>-1</artifactDaysToKeep>
        <artifactNumToKeep>-1</artifactNumToKeep>
    </logRotator>
</project>
对应DSL：
job('example') {
    logRotator(2, 10, -1, -1)
}
*/


// 配置电子邮件提醒
def emailTrigger = {
    trigger {
        email {
            recipientList '$PROJECT_DEFAULT_RECIPIENTS'
            subject '$PROJECT_DEFAULT_SUBJECT'
            body '$PROJECT_DEFAULT_CONTENT'
            sendToDevelopers true
            sendToRequester false
            includeCulprits false
            sendToRecipientList true
        }
    }
}

job('example') {
    configure { project ->
        project / publishers << 'hudson.plugins.emailext.ExtendedEmailPublisher' {
              recipientList 'Engineering@company.com'
              configuredTriggers {
                  'hudson.plugins.emailext.plugins.trigger.FailureTrigger' emailTrigger
                  'hudson.plugins.emailext.plugins.trigger.FixedTrigger' emailTrigger
              }
              contentType 'default'
              defaultSubject '$DEFAULT_SUBJECT'
              defaultContent '$DEFAULT_CONTENT'
        }
    }
}
/*
对应XML：
<project>
    <publishers>
        <hudson.plugins.emailext.ExtendedEmailPublisher>
            <recipientList>Engineering@company.com</recipientList>
            <configuredTriggers>
                <hudson.plugins.emailext.plugins.trigger.FailureTrigger>
                    <email>
                        <recipientList>$PROJECT_DEFAULT_RECIPIENTS</recipientList>
                        <subject>$PROJECT_DEFAULT_SUBJECT</subject>
                        <body>$PROJECT_DEFAULT_CONTENT</body>
                        <sendToDevelopers>true</sendToDevelopers>
                        <sendToRequester>false</sendToRequester>
                        <includeCulprits>false</includeCulprits>
                        <sendToRecipientList>true</sendToRecipientList>
                    </email>
                </hudson.plugins.emailext.plugins.trigger.FailureTrigger>
                <hudson.plugins.emailext.plugins.trigger.FixedTrigger>
                    <email>
                        <recipientList>$PROJECT_DEFAULT_RECIPIENTS</recipientList>
                        <subject>$PROJECT_DEFAULT_SUBJECT</subject>
                        <body>$PROJECT_DEFAULT_CONTENT</body>
                        <sendToDevelopers>true</sendToDevelopers>
                        <sendToRequester>false</sendToRequester>
                        <includeCulprits>true</includeCulprits>
                        <sendToRecipientList>true</sendToRecipientList>
                    </email>
                </hudson.plugins.emailext.plugins.trigger.FixedTrigger>
            </configuredTriggers>
            <contentType>default</contentType>
            <defaultSubject>$DEFAULT_SUBJECT</defaultSubject>
            <defaultContent>$DEFAULT_CONTENT</defaultContent>
        </hudson.plugins.emailext.ExtendedEmailPublisher>
    </publishers>
</project>
对应DSL：
job('example') {
    publishers {
        extendedEmail('Engineering@company.com') {
            trigger(triggerName: 'Failure', recipientList: '$PROJECT_DEFAULT_RECIPIENTS')
            trigger(triggerName: 'Fixed', recipientList: '$PROJECT_DEFAULT_RECIPIENTS')
        }
    }
}
*/


// 为Job添加Shell Step
job('example') {
    configure { project ->
        project / builders / 'hudson.tasks.Shell' {
            command 'echo "Hello" > ${WORKSPACE}/out.txt'
        }
    }
}
/*
对应XML：
<project>
    <builders>
        <hudson.tasks.Shell>
            <command>echo "Hello" > ${WORKSPACE}/out.txt</command>
        </hudson.tasks.Shell>
    </builders>
</project>
对应DSL：
job('example') {
    steps {
        shell 'echo "Hello" > ${WORKSPACE}/out.txt'
    }
}
*/


// 配置Gradle
job('example') {
    configure { project ->
        project / builders << 'hudson.plugins.gradle.Gradle' {
            description ''
            switches '-Dtiming-multiple=5'
            tasks 'test'
            rootBuildScriptDir ''
            buildFile ''
            useWrapper 'true'
            wrapperScript 'gradlew'
        }
    }
}
/*
对应XML：
<project>
    <builders>
        <hudson.plugins.gradle.Gradle>
            <description/>
            <switches>-Dtiming-multiple=5</switches>
            <tasks>test</tasks>
            <rootBuildScriptDir/>
            <buildFile/>
            <useWrapper>true</useWrapper>
            <wrapperScript>gradlew</wrapperScript>
        </hudson.plugins.gradle.Gradle>
    </builders>
</project>
对应DSL：
job('example') {
    steps {
        gradle('test', '-Dtiming-multiple-5', true) {
            it / wrapperScript 'gradlew'
        }
    }
}
*/


// 配置SVN
job('example') {
    configure { project ->
        project.remove(project / scm) // remove the existing 'scm' element
        project / scm(class: 'hudson.scm.SubversionSCM') {
            locations {
                'hudson.scm.SubversionSCM_-ModuleLocation' {
                    remote 'http://svn.apache.org/repos/asf/tomcat/maven-plugin/trunk'
                    local '.'
                }
            }
            excludedRegions ''
            includedRegions ''
            excludedUsers ''
            excludedRevprop ''
            excludedCommitMessages ''
            workspaceUpdater(class: "hudson.scm.subversion.UpdateUpdater")
        }
    }
}
/*
对应XML：
<project>
    <scm class="hudson.scm.SubversionSCM">
        <locations>
            <hudson.scm.SubversionSCM_-ModuleLocation>
                <remote>http://svn.apache.org/repos/asf/tomcat/maven-plugin/trunk</remote>
                <local>.</local>
            </hudson.scm.SubversionSCM_-ModuleLocation>
        </locations>
        <excludedRegions/>
        <includedRegions/>
        <excludedUsers/>
        <excludedRevprop/>
        <excludedCommitMessages/>
        <workspaceUpdater class="hudson.scm.subversion.UpdateUpdater"/>
    </scm>
</project>
对应DSL：
job('example') {
    scm {
        svn('http://svn.apache.org/repos/asf/tomcat/maven-plugin/trunk')
    }
}
*/


// 配置GIT
def gitConfigWithSubdir(subdir, remote) {
    { node ->
        // use remote name given
        node / 'userRemoteConfigs' / 'hudson.plugins.git.UserRemoteConfig' / name(remote)

        // use local dir given
        node / 'extensions' << 'hudson.plugins.git.extensions.impl.RelativeTargetDirectory' {
            relativeTargetDir subdir
        }

        // clean after checkout
        node / 'extensions' << 'hudson.plugins.git.extensions.impl.CleanCheckout'()
    }
}

job('example') {
    scm {
        git(
            'git@server:account/repo1.git',
            'remoteB/master',
            gitConfigWithSubdir('repo1', 'remoteB')
        )
    }
}
/*
对应XML：
<project>
    <scm class='hudson.plugins.git.GitSCM'>
        <userRemoteConfigs>
            <hudson.plugins.git.UserRemoteConfig>
                <url>git@server:account/repo1.git</url>
                <name>remoteB</name>
            </hudson.plugins.git.UserRemoteConfig>
        </userRemoteConfigs>
        <branches>
            <hudson.plugins.git.BranchSpec>
                <name>remoteB/master</name>
            </hudson.plugins.git.BranchSpec>
        </branches>
        <extensions>
            <hudson.plugins.git.extensions.impl.RelativeTargetDirectory>
                <relativeTargetDir>repo1</relativeTargetDir>
            </hudson.plugins.git.extensions.impl.RelativeTargetDirectory>
            <hudson.plugins.git.extensions.impl.CleanCheckout/>
        </extensions>
    </scm>
</project>
对应DSL：
job('example') {
    scm {
        git {
            remote {
                name 'remoteB'
                url 'git@server:account/repo1.git'
            }
            extensions {
                relativeTargetDirectory('repo1')
                cleanAfterCheckout()
            }
        }
    }
}
*/


// 配置Pre-requisite前置步骤
job('example') {
    configure { project ->
        project / builders / 'dk.hlyh.ciplugins.prereqbuildstep.PrereqBuilder' {
            projects('project-A,project-B')
            warningOnly(false)
        }
    }
}
/*
对应XML：
<project>
    <builders>
        <dk.hlyh.ciplugins.prereqbuildstep.PrereqBuilder>
            <projects>project-A,project-B</projects>
            <warningOnly>false</warningOnly>
        </dk.hlyh.ciplugins.prereqbuildstep.PrereqBuilder>
    </builders>
</project>
对应DSL：
job('example') {
    steps {
        prerequisite('project-A, project-B')
    }
}
*/


// 配置块重用
def switchOn = {
    it / 'properties' / 'com.example.Test' {
        'switch'('on')
    }
}
job('example-1') {
    configure switchOn
}

Closure switchOnOrOff(String value) {
    return {
        it / 'properties' / 'com.example.Test' {
            'switch'(value)
        }
    }
}
job('example-1') {
    configure switchOnOrOff('on')
}


// 根据属性选择子元素、添加孙子元素两个
job('example') {
    configure {
        def scm = it / scm(class: 'org.MyScm')
        scm << 'aChild' {
            serverUrl('http://example.org/product-a')
        }
        scm << 'aChild' {
            serverUrl('http://example.org/product-b')
        }
    }
}
/*
<project>
    <scm class='org.MyScm'>
        <aChild>
            <serverUrl>http://example.org/product-a</serverUrl>
        </aChild>
        <aChild>
            <serverUrl>http://example.org/product-b</serverUrl>
        </aChild>
    </scm>
</project>
*/

使用此变量可以获得被执行的DSL脚本的位置： 

println("script directory: ${new File(__FILE__).parent.absolutePath}")

通过此变量可以访问运行当前DSL的种子任务： 

job('example') {
    quietPeriod(SEED_JOB.quietPeriod)
}

有了job-dsl-plugin之后，你仍然需要手工登陆到Jenkins UI来创建种子任务。可以考虑安装Jenkins Configuration as Code插件，该插件允许你编写YAML来配置Jenkins。

下面是JCasC配置Jenkins安全的片段：

jenkins:
  securityRealm:
    ldap:
      configurations:
        - groupMembershipStrategy:
            fromUserRecord:
              attributeName: "memberOf"
          inhibitInferRootDN: false
          rootDN: "dc=acme,dc=org"
          server: "ldaps://ldap.acme.org:1636"

1. 安装Jenkins和本插件
2. 配置环境变量CASC_JENKINS_CONFIG，它可以是：
   1. 指向包含一系列配置文件的目录，例如/var/jenkins_home/casc_configs
   2. 指向配置文件
   3. 指向URL
3. 在Manage Jenkins ⇨ Configuration as Code管理本插件

本节给出一个样例，更多的配置示例参考官方文档。

jenkins:
  # 安全配置
  securityRealm:
    ldap:
      configurations:
        - groupMembershipStrategy:
            fromUserRecord:
              attributeName: "memberOf"
          inhibitInferRootDN: false
          rootDN: "dc=acme,dc=org"
          server: "ldaps://ldap.acme.org:1636"
  # 节点配置
  nodes:
    - permanent:
        name: "static-agent"
        remoteFS: "/home/jenkins"
        launcher:
          jnlp:

  slaveAgentPort: 50000
  agentProtocols:
    - "jnlp2"
# 开发工具配置
tool:
  git:
    installations:
      - name: git
        home: /usr/local/bin/git
unclassified:
  mailer:
    adminAddress: admin@acme.org
    replyToAddress: do-not-reply@acme.org
    # Note that this does not work right now
    #smtpHost: smtp.acme.org
    smtpPort: 4441

# 凭证信息
credentials:
  system:
    domainCredentials:
      credentials:
        - certificate:
            scope: SYSTEM
            id: ssh_private_key
            keyStoreSource:
              fileOnMaster:
                keyStoreFile: /docker/secret/id_rsa

触发Jenkins Reload配置的方式有：

1. 通过Jenkins UI： Manage Jenkins ⇨ Configuration ⇨ Reload existing configuration
2. 通过Jenkins CLI
3. 发送POST请求到JENKINS_URL/configuration-as-code/reload，可能需要凭证信息

这是一个Operator，能够在K8S上对Jenkins进行全面的管理。特性包括：

1. Pipeline as Code
2. 支持基于Groovy脚本或者JCasC进行扩展
3. 安全加固

首先需要安装CRD：

# kubectl apply -f https://raw.githubusercontent.com/jenkinsci/kubernetes-operator/master/deploy/crds/jenkins_v1alpha2_jenkins_crd.yaml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: jenkins.jenkins.io
spec:
  group: jenkins.io
  names:
    kind: Jenkins
    listKind: JenkinsList
    plural: jenkins
    singular: jenkins
  scope: Namespaced
  versions:
    - name : v1alpha2
      served: true
      storage: true
    - name : v1alpha1
      served: true
      storage: false

然后安装此CRD的Operator：

# kubectl apply -f https://raw.githubusercontent.com/jenkinsci/kubernetes-operator/master/deploy/all-in-one-v1alpha2.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkins-operator
spec:
  replicas: 1
  selector:
    matchLabels:
      name: jenkins-operator
  template:
    metadata:
      labels:
        name: jenkins-operator
    spec:
      serviceAccountName: jenkins-operator
      containers:
        - name: jenkins-operator
          image: virtuslab/jenkins-operator:v0.1.0
          command:
          - jenkins-operator
          args: []
          imagePullPolicy: IfNotPresent
          env:
            - name: WATCH_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: OPERATOR_NAME
              value: "jenkins-operator"

要安装一个Jenkins服务，只需要创建Jenkins类型的CR即可，Jenkins Operator会自动完成其它工作：

apiVersion: jenkins.io/v1alpha2
kind: Jenkins
metadata:
  name: example
spec:
  master:
    containers:
    - name: jenkins-master
      image: jenkins/jenkins:lts
      imagePullPolicy: Always
      livenessProbe:
        failureThreshold: 12
        httpGet:
          path: /login
          port: http
          scheme: HTTP
        initialDelaySeconds: 80
        periodSeconds: 10
        successThreshold: 1
        timeoutSeconds: 5
      readinessProbe:
        failureThreshold: 3
        httpGet:
          path: /login
          port: http
          scheme: HTTP
        initialDelaySeconds: 30
        periodSeconds: 10
        successThreshold: 1
        timeoutSeconds: 1
      resources:
        limits:
          cpu: 1500m
          memory: 3Gi
        requests:
          cpu: "1"
          memory: 500Mi
  seedJobs:
  - id: jenkins-operator
    targets: "cicd/jobs/*.jenkins"
    description: "Jenkins Operator repository"
    repositoryBranch: master
    repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git

等Jenkins的Pod可用后，执行下面的命令获取凭证信息：

kubectl get secret jenkins-operator-credentials-example -o 'jsonpath={.data.user}' | base64 -d
kubectl get secret jenkins-operator-credentials-example -o 'jsonpath={.data.password}' | base64 -d

Jenkins Operator依赖以下插件：

1. job-dsl-plugin，使用该插件提供的“种子任务”、DSL
2.  kubernetes-credentials-provider，在K8S中配置部署密钥

默认情况下Jenkins Operator期望你的项目下有如下目录：

cicd/
├── jobs
│   └── build.jenkins
└── pipelines
    └── build.jenkins

其中jobs包含Jenkins Job的定义，基于Job DSL语法：

#!/usr/bin/env groovy

// 定义一个流水线任务
pipelineJob('build-jenkins-operator') {
    // 显示名称
    displayName('Build jenkins-operator')
    // 任务定义
    definition {
        cpsScm {
            // 代码库信息
            scm {
                git {
                    remote {
                        url('https://github.com/jenkinsci/kubernetes-operator.git')
                        credentials('jenkins-operator')
                    }
                    branches('*/master')
                }
            }
            // 引用实际构建流水线
            scriptPath('cicd/pipelines/build.jenkins')
        }
    }
}

而pipelines则存放实际的构建流水线，例如： 

#!/usr/bin/env groovy

def label = "build-jenkins-operator-${UUID.randomUUID().toString()}"
def home = "/home/jenkins"
def workspace = "${home}/workspace/build-jenkins-operator"
def workdir = "${workspace}/src/github.com/jenkinsci/kubernetes-operator/"

// 在此模板所定义的Pod中执行流水线
podTemplate(label: label,
        containers: [
                // 必须有一个名为jnlp的容器，负责和Jenkins服务器的交互
                containerTemplate(name: 'jnlp', image: 'jenkins/jnlp-slave:alpine'),
                // 这个容器则负责运行构建工具
                containerTemplate(name: 'go', image: 'golang:1-alpine', command: 'cat', ttyEnabled: true),
        ],
        envVars: [
                envVar(key: 'GOPATH', value: workspace),
        ],
        ) {
    // 在上述Pod中
    node(label) {
        // 切换工作目录
        dir(workdir) {
            // 然后在go容器中完成构建
            stage('Init') {
                timeout(time: 3, unit: 'MINUTES') {
                    checkout scm
                }
                container('go') {
                    sh 'apk --no-cache --update add make git gcc libc-dev'
                }
            }

            stage('Dep') {
                container('go') {
                    sh 'make dep'
                }
            }

            stage('Test') {
                container('go') {
                    sh 'make test'
                }
            }

            stage('Build') {
                container('go') {
                    sh 'make build'
                }
            }
        }
    }
}

除了在工程目录中提供上述种子任务、流水线定义，你还需要配置Jenkins CR，告知Operator到什么地方获取种子任务、流水线： 

apiVersion: jenkins.io/v1alpha2
kind: Jenkins
metadata:
  name: example
spec:
  seedJobs:
  - id: jenkins-operator
    targets: "cicd/jobs/*.jenkins"
    description: "Jenkins Operator repository"
    repositoryBranch: master
    repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git
  - id: jenkins-operator-ssh
    # 如果Git仓库是私有的，可以使用基于SSH的身份验证
    credentialType: basicSSHUserPrivateKey
    credentialID: k8s-ssh
    # 也可以使用基于用户名密码的身份验证
  - id: jenkins-operator-user-pass
    credentialType: usernamePassword
    credentialID: k8s-user-pass

# 上述k8s-ssh、k8s-user-pass必须配置为K8S Secret：

apiVersion: v1
kind: Secret
metadata:
  name: k8s-ssh
data:
  privateKey: |
    -----BEGIN RSA PRIVATE KEY-----
    MIIJKAIBAAKCAgEAxxDpleJjMCN5nusfW/AtBAZhx8UVVlhhhIKXvQ+dFODQIdzO
    oDXybs1zVHWOj31zqbbJnsfsVZ9Uf3p9k6xpJ3WFY9b85WasqTDN1xmSd6swD4N8
    ...
  username: github_user_name


apiVersion: v1
kind: Secret
metadata:
  name: k8s-user-pass
data:
  username: github_user_name
  password: password_or_token

要为Jenkins安装插件，可以配置CR，增加spec.master.plugins字段：

apiVersion: jenkins.io/v1alpha2
kind: Jenkins
metadata:
  name: example
spec:
  master:
   plugins:
   - name: simple-theme-plugin
     version: 0.5.1

定制Jenkins配置，可以使用Groovy脚本或者JCasC。所有定制配置信息都存放在Secret  jenkins-operator-user-configuration-<cr_name>中：

# kubectl get configmap jenkins-operator-user-configuration-<cr_name> -o yaml

apiVersion: v1
data:
  # 使用Groovy脚本陪孩子
  1-configure-theme.groovy: |2
    import jenkins.*
    import jenkins.model.*
    import hudson.*
    import hudson.model.*
    import org.jenkinsci.plugins.simpletheme.ThemeElement
    import org.jenkinsci.plugins.simpletheme.CssTextThemeElement
    import org.jenkinsci.plugins.simpletheme.CssUrlThemeElement
    
    # 获取Jenkins实例对象
    Jenkins jenkins = Jenkins.getInstance()

    # 获取插件配置对象
    def decorator = Jenkins.instance.getDescriptorByType(org.codefirst.SimpleThemeDecorator.class)

    List<ThemeElement> configElements = new ArrayList<>();
    configElements.add(new CssTextThemeElement("DEFAULT"));
    configElements.add(new CssUrlThemeElement("https://cdn.rawgit.com/afonsof/jenkins-material-theme/gh-pages/dist/material-light-green.css"));
    decorator.setElements(configElements);
    decorator.save();
    # 保存配置
    jenkins.save()
  # 使用JCasC也可以
  1-system-message.yaml: |2
    jenkins:
      systemMessage: "Configuration as Code integration works!!!"
      adminAddress: "${SECRET_JENKINS_ADMIN_ADDRESS}"
kind: ConfigMap
metadata:
  name: jenkins-operator-user-configuration-<cr_name>
  namespace: default

本文调研的这些技术中，除了Jenkins REST API以外都是面向Jenkins的最终用户的，不适合进行二次开发。

job-dsl-plugin提供的DSL的确具有其价值，它简化了Jenkins API的复杂度，可以避免编写繁琐的XML操控代码。所以，通过Jenkins REST API创建基于job-dsl-plugin的种子任务，并立即触发来生成Jenkins流水线任务是一种可行的方案。

对于最终用户来说，job-dsl-plugin的DSL还是太过复杂，应当进一步简化、屏蔽技术细节。

The post 在Kubernetes中管理和使用Jenkins appeared first on 绿色记忆.

Byte Buddy是一个JVM的运行时代码生成器，你可以利用它创建任何类，且不像JDK动态代理那样强制实现一个接口。Byte Buddy还提供了简单的API，便于手工、通过Java Agent，或者在构建期间修改字节码。

Java反射API可以做很多和字节码生成器类似的工作，但是它具有以下缺点：

1. 相比硬编码的方法调用，使用 反射 API 非常慢
2. 反射 API 能绕过类型安全检查

比起JDK动态代理、cglib、Javassist，Byte Buddy在性能上具有优势。

下面是一个最简单的例子：

Class<?> dynamicType = new ByteBuddy()
  // 指定父类
  .subclass(Object.class)
   // 根据名称来匹配需要拦截的方法
  .method(ElementMatchers.named("toString"))
  // 拦截方法调用，返回固定值
  .intercept(FixedValue.value("Hello World!"))
  // 产生字节码
  .make()
  // 加载类
  .load(getClass().getClassLoader())
  // 获得Class对象
  .getLoaded();

assertThat(dynamicType.newInstance().toString(), is("Hello World!"));

ByteBuddy利用Implementation接口来表示一个动态定义的方法，FixedValue.value就是该接口的实例。

完全实现Implementation比较繁琐，因此实际情况下会使用MethodDelegation代替。使用MethodDelegation，你可以在一个POJO中实现方法拦截器：

public class GreetingInterceptor {
  // 方法签名随意
  public Object greet(Object argument) {
    return "Hello from " + argument;
  }
}

Class<? extends java.util.function.Function> dynamicType = new ByteBuddy()
  // 实现一个Function子类
  .subclass(java.util.function.Function.class)
  .method(ElementMatchers.named("apply"))
  // 拦截Function.apply调用，委托给GreetingInterceptor处理
  .intercept(MethodDelegation.to(new GreetingInterceptor()))
  .make()
  .load(getClass().getClassLoader())
  .getLoaded();

assertThat((String) dynamicType.newInstance().apply("Byte Buddy"), is("Hello from Byte Buddy"));

编写拦截器时，你可以指定一些注解，ByteBuddy会自动注入：

public class GeneralInterceptor {
  // 提示ByteBuddy根据被拦截方法的实际类型，对此拦截器的返回值进行Cast
  @RuntimeType
  //                      所有入参的数组
  public Object intercept(@AllArguments Object[] allArguments,
  //                      被拦截的原始方法
                          @Origin Method method) {
  }
}

上面的两个例子中，我们利用ByteBuddy创建了指定接口的新子类型，ByteBuddy也可以用来修改已存在的。

ByteBuddy提供了便捷的创建Java Agent的API，本节的例子就是通过Java Agent方式来修改已存在的Java类型：public class TimerAgent {

public static void premain(String arguments, 
                             Instrumentation instrumentation) {
    new AgentBuilder.Default()
      // 匹配被拦截方法
      .type(ElementMatchers.nameEndsWith("Timed"))
      .transform(
          (builder, type, classLoader, module) -> 
              builder.method(ElementMatchers.any()) .intercept(MethodDelegation.to(TimingInterceptor.class))
      ).installOn(instrumentation);
  }
}

public class TimingInterceptor {
  @RuntimeType
  public static Object intercept(@Origin Method method, 
                                 // 调用该注解后的Runnable/Callable，会导致调用被代理的非抽象父方法
                                 @SuperCall Callable<?> callable) {
    long start = System.currentTimeMillis();
    try {
      return callable.call();
    } finally {
      System.out.println(method + " took " + (System.currentTimeMillis() - start));
    }
  }
}

调用此方法可以创建一个目标类的子类：

DynamicType.Unloaded<?> dynamicType = new ByteBuddy()
  .subclass(Object.class)
  .name("example.Type")  // 子类的名称
  .make();

如果不指定子类名称，Byte Buddy会有一套自动的策略来生成。你还可以指定子类命名策略：

DynamicType.Unloaded<?> dynamicType = new ByteBuddy()
  .with(new NamingStrategy.AbstractBase() {
    @Override
    public String subclass(TypeDescription superClass) {
        return "i.love.ByteBuddy." + superClass.getSimpleName();
    }
  })
  .subclass(Object.class)
  .make();

上节创建的DynamicType.Unloaded，代表一个尚未加载的类，你可以通过ClassLoadingStrategy来加载这种类。 

如果不指定ClassLoadingStrategy，Byte Buffer根据你提供的ClassLoader来推导出一个策略，内置的策略定义在枚举ClassLoadingStrategy.Default中：

1. WRAPPER：创建一个新的Wrapping类加载器
2. CHILD_FIRST：类似上面，但是子加载器优先负责加载目标类
3. INJECTION：利用反射机制注入动态类型

示例：

Class<?> type = new ByteBuddy()
  .subclass(Object.class)
  .make()
  .load(getClass().getClassLoader(), ClassLoadingStrategy.Default.WRAPPER)
  .getLoaded();

重定义一个类时，Byte Buddy 可以对一个已有的类添加属性和方法，或者删除已经存在的方法实现。新添加的方法，如果签名和原有方法一致，则原有方法会消失。

类似于redefine，但是原有的方法不会消失，而是被重命名，添加后缀

$original

class Foo {
  String bar() { return "bar"; }
}

在rebase之后，会变成：

class Foo {
  String bar() { return "foo" + bar$original(); }
  private String bar$original() { return "bar"; }
}

得益于JVM的HostSwap特性，已加载的类可以被重新定义：

// 安装Byte Buddy的Agent，除了通过-javaagent静态安装，还可以：
ByteBuddyAgent.install();
Foo foo = new Foo();
new ByteBuddy()
  .redefine(Bar.class)
  .name(Foo.class.getName())
  .make()
  .load(Foo.class.getClassLoader(), ClassReloadingStrategy.fromInstalledAgent());

assertThat(foo.m(), is("bar"));

可以看到，即使时已经存在的对象，也会受到类Reloading的影响。

当前HostSwap具有限制：

1. 类再重新载入前后，必须具有相同的Schema，也就是方法、字段不能减少（可以增加）
2. 不支持具有静态初始化块的类

Byte Buddy提供了类似于Javassist的、操控未加载类的API。它在TypePool中维护类型的元数据TypeDescription：

// 获取默认类型池
TypePool typePool = TypePool.Default.ofClassPath();
new ByteBuddy()
  .redefine(typePool.describe("foo.Bar").resolve(), // 根据名称进行解析类
            // ClassFileLocator用于定位到被修改类的.class文件
            ClassFileLocator.ForClassLoader.ofClassPath())
  .defineField("qux", String.class) // 定义一个新的字段
  .make()
  .load(ClassLoader.getSystemClassLoader());
assertThat(Bar.class.getDeclaredField("qux"), notNullValue());

Byte Buddy提供了很多用于匹配方法的DSL：

class Foo {
  public String bar() { return null; }
  public String foo() { return null; }
  public String foo(Object o) { return null; }
}
 
Foo dynamicFoo = new ByteBuddy()
  .subclass(Foo.class)
  // 匹配由Foo.class声明的方法
  .method(isDeclaredBy(Foo.class)).intercept(FixedValue.value("One!"))
  // 匹配名为foo的方法
  .method(named("foo")).intercept(FixedValue.value("Two!"))
  // 匹配名为foo，入参数量为1的方法
  .method(named("foo").and(takesArguments(1))).intercept(FixedValue.value("Three!"))
  .make()
  .load(getClass().getClassLoader())
  .getLoaded()
  .newInstance();

使用MethodDelegation可以将方法调用委托给任意POJO。Byte Buddy不要求Source（被委托类）、Target类的方法名一致：

class Source {
  public String hello(String name) { return null; }
}
 
String helloWorld = new ByteBuddy()
  .subclass(Source.class)
  .method(named("hello")).intercept(MethodDelegation.to(Target.class))
  .make()
  .load(getClass().getClassLoader())
  .getLoaded()
  .newInstance()
  .hello("World");

Target的实现可以如下： 

class Target {
  public static String hello(String name) {
    return "Hello " + name + "!";
  }
} 

也可以如下：  

class Target {
  public static String intercept(String name) { return "Hello " + name + "!"; }
  public static String intercept(int i) { return Integer.toString(i); }
  public static String intercept(Object o) { return o.toString(); }
}

前一个实现很好理解，那么后一个呢，Byte Buddy到底会委托给哪个方法？Byte Buddy遵循一个最接近原则：

1. intercept(int)因为参数类型不匹配，直接Pass
2. 另外两个方法参数都匹配，但是 intercept(String)类型更加接近，因此会委托给它

你可以在Target的方法中使用注解进行参数绑定：

void foo(Object o1, Object o2)
// 等价于
void foo(@Argument(0) Object o1, @Argument(1) Object o2)

全部注解如下表：

Class<? extends UserType> dynamicUserType = new ByteBuddy()
  .subclass(UserType.class)
  .defineField("interceptor", Interceptor.class, Visibility.PRIVATE);

方法调用也可以委托给字段（而非外部对象）：

Class<? extends UserType> dynamicUserType = new ByteBuddy()
  .subclass(UserType.class)
    .method(not(isDeclaredBy(Object.class)))
    .intercept(MethodDelegation.toField("interceptor")); 

The post Byte Buddy学习笔记 appeared first on 绿色记忆.

此API由java.lang.instrument包提供，其核心是Instrumentation接口，它提供了探测（instrument）Java代码的基本服务，可用于实现性能监控、Profiler、事件记录器等功能。获取Instrumentation实例的方法有两种：

1. 如果JVM启动时静态加载了Agent，那么Instrumentation被传递给Agent类的

   premain

   方法
2. 如果JVM在运行时动态加载了Agent，那么Instrumentation被传递给Agent类的

   agentmain

   方法

Instrumentation接口最常用的方法包括：

Java Agent本质上就是一个Jar包，它会调用 Instrumentation API，来修改已经加载到JVM中的字节码。Java Agent具有两种加载方式。

这种方式下，在应用程序的任何代码被执行之前，就加载Agent以修改字节码。静态加载需要使用JVM的-javaagent参数：

java -javaagent:agent.jar -jar application.jar
# 可以同时加载多个Agents
java -javaagent:agentA.jar -javaagent:agentB.jar application.jar

这种方式下，Agent可以在运行时动态按需的加载。动态加载需要调用Java Attach API，下面是个例子：

// 根据PID查找目标JVM，并连接到JVM
VirtualMachine jvm = VirtualMachine.attach(jvmPid);
// 加载Agent
jvm.loadAgent(agentFile.getAbsolutePath());
// 取消到JVM的连接
jvm.detach();

Agent类就是一个普通的Java类，不需要特殊签名或者实现接口。根据加载方式的不同，你需要添加下面的方法：

public static void premain(
  String agentArgs, Instrumentation inst) {
    LOGGER.info("[Agent] In premain method");
    String className = "com.baeldung.instrumentation.application.MyAtm";
    // 添加转换器
    transformClass(className,inst);
}
public static void agentmain(
  String agentArgs, Instrumentation inst) {
  
    LOGGER.info("[Agent] In agentmain method");
    String className = "com.baeldung.instrumentation.application.MyAtm";
    transformClass(className,inst);
}

转换器实现ClassFileTransformer接口的transform方法，进行字节码编辑。我们通常会利用javassist来操控字节码。

下面的例子，修改HttpURLConnection类，以便对JVM访问的每个URL进行审计：

public class MyClassTransformer implements ClassFileTransformer {
  @Override
  public byte[] transform( 
      final ClassLoader loader, 
      final String className,
      final Class<?> classBeingRedefined, 
      final ProtectionDomain protectionDomain,
      final byte[] classfileBuffer ) throws IllegalClassFormatException {
    // 仅仅操作HttpURLConnection类
    if (className.endsWith("sun/net/www/protocol/http/HttpURLConnection")) {
      try {
        // 从ClassPool获得CtClass对象
        final ClassPool classPool = ClassPool.getDefault();
        final CtClass clazz = classPool.get("sun.net.www.protocol.http.HttpURLConnection");
        // 修改构造函数，在其结尾添加日志记录逻辑    
        for (final CtConstructor constructor: clazz.getConstructors()) {
          constructor.insertAfter("System.out.println(this.getURL());");
        }
        // 返回字节码，并且detachCtClass对象
        byte[] byteCode = clazz.toBytecode();
        clazz.detach();
              
        return byteCode;
      } catch (final NotFoundException | CannotCompileException | IOException ex) {
        ex.printStackTrace();
      }
    }
    // 如果返回null则字节码不会被修改
    return null;
  }
}

你需要在Jar的MANIFEST.MF文件中适当的属性：

Agent-Class: cc.gmem.agent.MyInstrumentationAgent
Can-Redefine-Classes: true
Can-Retransform-Classes: true
Premain-Class: cc.gmem.agent.MyInstrumentationAgent

本章提供一个例子，该例子没有直接使用Instrumentation API，而是调用Byte Buddy提供的更简便的API。 在该例子中，我们拦截IntelliJ IDE的DesktopLayout.getInfo方法，实现Tool Window的宽度、高度固定化。

package cc.gmem.intellij;

import net.bytebuddy.agent.builder.AgentBuilder;
import net.bytebuddy.implementation.MethodDelegation;

import java.lang.instrument.Instrumentation;

import static net.bytebuddy.matcher.ElementMatchers.named;
import static net.bytebuddy.matcher.ElementMatchers.takesArguments;

public class IntellijUIAgent {

    public static void premain( String arguments, Instrumentation instrumentation ) throws Exception {
        new AgentBuilder.Default()
            // 需要拦截的类
            .type( named( "com.intellij.openapi.wm.impl.DesktopLayout" ) )
            .transform( ( builder, type, classLoader, module ) -> {
                // 需要拦截的方法
                return builder.method( named( "getInfo" ).and( takesArguments( String.class, boolean.class ) ) )
                              // to可以指定一个对象，也可以指定一个类。后者的话，拦截器方法必须是static
                              .intercept( MethodDelegation.withDefaultConfiguration().to( createaInterceptor( classLoader, arguments ) ) );
            } )
            // 将无法拦截的错误信息打印到控制台
            .with( AgentBuilder.Listener.StreamWriting.toSystemOut().withErrorsOnly() )
            // 使用REBASE方式，否则可能出现None of [XXX] allows for delegation from YYY
            .with( AgentBuilder.TypeStrategy.Default.REBASE )
            .installOn( instrumentation );
        System.out.println( "Intellij UI Agent by Alex Wong" );
    }
    // 注意不要通过new操作符直接创建DesktopLayoutInterceptor，原因是
    // Intellij使用独立的ClassLoader com.intellij.util.lang.UrlClassLoader来加载它自己的类库
    // 你new的时候使用的是系统类加载器
    private static Object createaInterceptor( ClassLoader classLoader, String arguments ) {
        try {
            String[] weights = arguments.split( "," );
            float leftWeight = Float.parseFloat( weights[0] );
            float bottomWeight = Float.parseFloat( weights[1] );
            Class<?> cls = classLoader.loadClass( "cc.gmem.intellij.DesktopLayoutInterceptor" );
            return cls.getConstructor( float.class, float.class ).newInstance( leftWeight, bottomWeight );
        } catch ( Exception e ) {
            throw new RuntimeException( e );
        }
    }
}

package cc.gmem.intellij;

import com.intellij.openapi.wm.ToolWindowAnchor;
import com.intellij.openapi.wm.impl.WindowInfoImpl;
import net.bytebuddy.implementation.bind.annotation.Origin;
import net.bytebuddy.implementation.bind.annotation.RuntimeType;
import net.bytebuddy.implementation.bind.annotation.SuperCall;

import java.lang.reflect.Method;
import java.util.concurrent.Callable;

public class DesktopLayoutInterceptor {

    private final float leftWeight;

    private final float bottomWeight;

    public DesktopLayoutInterceptor( float leftWeight, float bottomWeight ) {
        this.leftWeight = leftWeight;
        this.bottomWeight = bottomWeight;
    }

    @RuntimeType
    public Object getInfo( @Origin Method method, @SuperCall Callable<?> callable ) {
        WindowInfoImpl info = null;
        try {
            info = (WindowInfoImpl) callable.call();
        } catch ( Exception e ) {
            e.printStackTrace();
        }
        if ( info.getAnchor() == ToolWindowAnchor.LEFT ) {
            info.setWeight( leftWeight );
        } else if ( info.getAnchor() == ToolWindowAnchor.BOTTOM ) {
            info.setWeight( bottomWeight );
        }
        return info;
    }
}

# 注意向Agent传参的方式
-javaagent:target/ui-agent-1.0-SNAPSHOT.jar=0.28,0.4

The post 如何开发Java Agent appeared first on 绿色记忆.

SOFAStack（Scalable Open Financial Architecture Stack，可扩展开放金融架构栈）是蚂蚁金服开源的技术栈，国内多家金融和互联网公司在生产环境使用了此技术栈。

基于Spring Boot，额外提供了以下特性：

1. 健康检查（Readiness探针）：在Spring Boot Liveness探针的基础上增加Readiness探针，仅当此探针成功后实例才能接受流量
2. 类（加载器）隔离，基于SOFAArk，实现业务代码的类、SOFA中间件相关的类的隔离，避免了类冲突
3. 日志空间隔离：将SOFA中间件日志和业务代码产生的日志分开
4. 和其他SOFA中间件便捷的集成：提供各种SOFA中间件的Starter
5. 模块化：可以为同一JVM中运行的不同SOFABoot模块提供独立的Spring ApplicationContext，可以规避BeanId的冲突

SOFABoot需要JDK 7+和Maven 3.2.5来完成构建。

通过IntelliJ IDEA的Spring Initializer来创建项目骨架，依赖选择Web。然后修改POM，将parent改为：

<parent>
    <groupId>com.alipay.sofa</groupId>
    <artifactId>sofaboot-dependencies</artifactId>
    <version>${sofa.boot.version}</version>
</parent>

并添加依赖： 

<!-- 提供健康检查能力 -->
<dependency>
    <groupId>com.alipay.sofa</groupId>
    <artifactId>healthcheck-sofa-boot-starter</artifactId>
</dependency>

<dependency>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-starter-web</artifactId>
</dependency>

SOFABoot要求提供必要的参数：

spring.application.name=Learing SOFABoot
logging.path=./logs

启动上述项目后，运行命令

curl http://localhost:8080/actuator/health

{"status":"UP"}

运行命令

curl -s http://localhost:8080/actuator/readiness | jq .

{
  "details": {
    "diskSpace": {
      "details": {
        "threshold": 10485760,
        "free": 166560387072,
        "total": 358796750848
      },
      "status": "UP"
    },
    "SOFABootReadinessHealthCheckInfo": {
      "status": "UP"
    }
  },
  "status": "UP"
}

检查logs目录，可以看到日志根据来源的不同，分割到logs、infra等目录中。 

你可以使用SpringRunner进行单元测试，但是如果在项目中使用了SOFABoot的类隔离特性，则必须使用SofaBootRunner、SofaJUnit4Runner进行单元测试，并引入依赖：

<dependency>
    <groupId>com.alipay.sofa</groupId>
    <artifactId>test-sofa-boot-starter</artifactId>
</dependency>

单元测试用例的例子：

@SpringBootTest
@RunWith(SpringRunner.class)
public class SofaBootWithModulesTest {
    // 引用SOFABoot模块发布的服务
    @SofaReference
    private SampleJvmService sampleJvmService;

    @Test
    public void test() {
        Assert.assertEquals("Hello, jvm service xml implementation.", sampleJvmService.message());
    }
} 

从2.4版本开始SOFABoot引入了基于Spring上下文隔离的模块化能力。SOFABoot定义了三个模块化隔离级别：

1. 代码组织上的模块化：开发阶段分多个工程开发，例如使用Dubbo的团队，通常都会将API拆分为独立的Maven模块。这种隔离对运行时没有任何影响
2. 基于Spring上下文隔离的模块化“类似于1，代码和配置分散在不同工程中。但是在运行时会启动多个Spring上下文（它们有共同的父上下文），DI仅仅在子上下文内发生。可以规避BeanId冲突问题
3. 基于类加载器隔离的模块化：每个模块都使用独立的类加载器，可以规避模块依赖了冲突的类版本的问题

SOFABoot模块化开发属于第二种模块化形式 —— 基于 Spring 上下文隔离的模块化，每个模块使用独立的Spring上下文。

每个SOFABoot模块包含Java代码、Spring配置文集、SOFA模块标识等信息，打包形式为JAR。不同模块之间不能通过DI来引用，需要转而使用SOFA服务。SOFABoot支持两种形式的服务（发布/引用）：

1. JVM服务发布和引用：解决同一SOFABoot 应用内各 SOFABoot 模块之间的调用问题
2. RPC服务发布和引用：解决多个 SOFABoot 应用之间的远程调用问题

由于相互之间没有Bean依赖，SOFABoot模块可以并行的启动，这可以提升应用启动速度。

这种模块化开发，和微服务的理念是违背的，各模块仍然需要共享单个JVM的资源。

为了体验SOFABoot的模块化开发，我们直接使用其源码附带的示例应用：

git clone https://github.com/alipay/sofa-boot.git
tree sofa-boot/sofaboot-samples/sofaboot-sample-with-isle -L 1
# .
# ├── service-consumer    服务的消费者
# ├── service-facade      服务的API
# ├── service-provider    服务的提供者
# ├── sofa-boot-run       启动包含模块的SOFABoot应用

service-facade中定义的接口如下：

public interface SampleJvmService {
    String message();
}

service-provider将上面的接口发布为JVM服务。发布方式有三种：

1. 注解方式：
   

   //           唯一性的ID，不设置默认为空串
   @SofaService(uniqueId = "annotationImpl")
   public class SampleJvmServiceAnnotationImpl implements SampleJvmService {
       @Override
       public String message() {
           String message = "Hello, jvm service annotation implementation.";
           System.out.println(message);
           return message;
       }
   }

2. XML方式： 

   public class SampleJvmServiceImpl implements SampleJvmService {
       private String message;
   
       @Override
       public String message() {
           System.out.println(message);
           return message;
       }
   }

   需要配合XML配置： 

   <bean id="sampleJvmService" class="com.alipay.sofa.isle.sample.SampleJvmServiceImpl">
       <property name="message" value="Hello, jvm service xml implementation."/>
   </bean>
   
   <sofa:service ref="sampleJvmService" interface="com.alipay.sofa.isle.sample.SampleJvmService">
       <sofa:binding.jvm/>
   </sofa:service>

3. 编程式：
   

   @Component
   public class PublishServiceWithClient implements ClientFactoryAware {
       private ClientFactory clientFactory;
       
       @PostConstruct
       public void init() {
           ServiceClient serviceClient = clientFactory.getClient(ServiceClient.class);
           ServiceParam serviceParam = new ServiceParam();
           serviceParam.setInstance(new SampleJvmServiceImpl( "Hello, jvm service service client implementation."));
           serviceParam.setInterfaceType(SampleJvmService.class);
           serviceParam.setUniqueId("serviceClientImpl");
           // 发布服务
           serviceClient.service(serviceParam);
       }
   
       @Override
       public void setClientFactory(ClientFactory clientFactory) {
           this.clientFactory = clientFactory;
       }
   } 

你需要在sofa-module.properties中声明模块名称、模块依赖：

# service-provider
Module-Name=com.alipay.sofa.service-provider

# service-consumer
Module-Name=com.alipay.sofa.service-consumer
Require-Module=com.alipay.sofa.service-provider

service-consumer负责消费service-provider发布的服务：

public class JvmServiceConsumer implements ClientFactoryAware {
    private ClientFactory    clientFactory;

    // 引用基于XML配置的JVM服务
    @Autowired
    private SampleJvmService sampleJvmService;
    // 引用基于注解配置的JVM服务，使用uniqueId
    @SofaReference(uniqueId = "annotationImpl")
    private SampleJvmService sampleJvmServiceByFieldAnnotation;

    public void init() {
        sampleJvmService.message();
        sampleJvmServiceByFieldAnnotation.message();

        // 编程式客户端
        ReferenceClient referenceClient = clientFactory.getClient(ReferenceClient.class);
        ReferenceParam referenceParam = new ReferenceParam<>();
        referenceParam.setInterfaceType(SampleJvmService.class);
        referenceParam.setUniqueId("serviceClientImpl");
        SampleJvmService sampleJvmServiceClientImpl = referenceClient.reference(referenceParam);
        sampleJvmServiceClientImpl.message();
    }

    @Override
    public void setClientFactory(ClientFactory clientFactory) {
        this.clientFactory = clientFactory;
    }
}

如果需要在SOFABoot项目中使用SOFA中间件，需要依赖相应的Starter： 

<!-- SOFARPC -->
<dependency>
    <groupId>com.alipay.sofa</groupId>
    <artifactId>rpc-sofa-boot-starter</artifactId>
</dependency>

<!-- SOFATracer -->
<dependency>
    <groupId>com.alipay.sofa</groupId>
    <artifactId>tracer-sofa-boot-starter</artifactId>
</dependency>

<!-- SOFALookout -->
<dependency>
    <groupId>com.alipay.sofa</groupId>
    <artifactId>lookout-sofa-boot-starter</artifactId>
</dependency>

 如果需要在SOFABoot项目中使用扩展组件，需要依赖相应的Starter： 

<!-- 健康检查扩展 -->
<dependency>
    <groupId>com.alipay.sofa</groupId>
    <artifactId>healthcheck-sofa-boot-starter</artifactId>
</dependency>


<!-- 模块化隔离扩展 -->
<dependency>
    <groupId>com.alipay.sofa</groupId>
    <artifactId>isle-sofa-boot-starter</artifactId>
</dependency>


<!-- 类隔离扩展 -->
<dependency>
    <groupId>com.alipay.sofa</groupId>
    <artifactId>sofa-ark-springboot-starter</artifactId>
</dependency>


<!-- 测试扩展 -->
<dependency>
    <groupId>com.alipay.sofa</groupId>
    <artifactId>test-sofa-boot-starter</artifactId>
</dependency>

如果要使用SOFAArk提供的类加载器隔离功能，则需要依赖相应的ARK插件，并替换到对应的Starter：

ARK插件能够让业务应用的类、SOFA中间件的类（以及它依赖的类）使用不同的类加载器，从而避免类冲突问题。

Spring Boot Actuator提供的HealthIndicator接口可以提供Liveness健康检查，SOFABoot在此基础上增加了Readiness检查功能。使用SOFA中间件时，最好配合SOFABoot的健康检查，以实现优雅上线，避免未准备好的实例过早加入服务池。

健康检查扩展提供了多个扩展点，允许用户定制其行为。

健康检查相关配置属性：

SOFABoot支持异步的执行Bean的初始化方法，要使用该特性，引入依赖：

<dependency>
    <groupId>com.alipay.sofa</groupId>
    <artifactId>runtime-sofa-boot-starter</artifactId>
</dependency>

并且为目标Bean提供配置属性async-init：

<bean id="testBean" class="com.alipay.sofa.runtime.beans.TimeWasteBean" init-method="init" async-init="true"/>

SOFABoot在独立的线程池中进行Bean的异步初始化，相关配置属性：

SOFABoot模块的JAR包遵循如下约定：

1. 包含文件sofa-module.properties，定义模块名称、模块之间的依赖关系等元数据
2. META-INF/spring目录下的任意Spring配置文件都会作为本模块的配置而加载

模块元数据声明在sofa-module.properties文件中，包含以下配置属性：

com.alipay.sofa.boot.active-profile

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:jdbc="http://www.springframework.org/schema/jdbc"
       xmlns:jee="http://www.springframework.org/schema/jee"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
         http://www.springframework.org/schema/beans/spring-beans-3.2.xsd
         http://www.springframework.org/schema/context
         http://www.springframework.org/schema/context/spring-context.xsd"
       default-autowire="byName">       

    <beans profile="dev">
        <bean id="devBeanId" class="com.alipay.cloudenginetest.sofaprofilesenv.DemoBean">
        </bean>
    </beans>

    <beans profile="test">
        <bean id="testBeanId" class="com.alipay.cloudenginetest.sofaprofilesenv.DemoBean">
        </bean>
    </beans>
</beans>

参考SOFARPC - 起步 - 整合SOFABoot一节。 

SOFAArk 是一款基于 Java 实现的轻量级类隔离容器，提供类隔离和应用（模块）合并部署能力。

在大型软件开发过程中，通常会推荐底层功能插件化，业务功能模块化的开发模式，以期达到低耦合、高内聚、功能复用的优点。SOFAArk 提供了一套插件化、模块化的开发规范，它的能力包括：

1. 定义类加载模型，运行时底层插件、业务应用（模块）的类相互隔离，每个插件和应用（模块）由不同的 ClassLoader 加载，可以有效避免相互之间的包冲突
2. 定义了插件开发规范，可以基于Maven将多个第三方JAR打包为Ark Plugin（插件）
3. 定义模块开发规范，可以基于Maven将应用打包为Ark Biz（模块）
4. 提供针对Plugin、Biz的标准API，提供事件、扩展点支持
5. 多Biz合并部署，打包为扁平的可执行JAR
6. 支持在运行时通过API或配置中心来动态安装/写在Biz

Ark 包是满足特定目录格式要求的可执行扁平（Flat，也就是打包了所有依赖）Jar，使用Maven 插件

sofa-ark-maven-plugin

1. Ark Container：负责 Ark 包启动、运行时的管理，Ark Plugin 和 Ark Biz 运行在 SOFAArk 容器之上。Ark Container具备管理插件和应用的能力，容器启动成功后，会自动解析类路径包含的 Ark Plugin 和 Ark Biz 依赖，完成隔离加载并按优先级依次启动它们
2. Ark Plugin，使用Maven插件

   sofa-ark-maven-plugin

   可以将单个或多个普通Jar包打包为Ark Plugin。Ark Plugin有一个配置文件，其中包含：插件类导入导出配置、资源导入导出配置、插件启动优先级等信息。SOFAArk使用独立的PluginClassLoader来加载插件
3. Ark Biz，使用Maven插件

   sofa-ark-maven-plugin

   可以将业务应用打包为Biz包。每个Ark包可以包含多个Biz，他们按优先级依次启动，通过JVM服务交互

执行Ark包时，Ark Container优先启动，然后启动Plugin，最后启动Biz。它们的逻辑关系图如下：

在解决类冲突（两个运行在单个JVM中的模块依赖某个类的两个不兼容版本）方面，SOFAArk比OSGI简单的多。SOFAArk 提出了一种特殊的包结构Ark Plugin，在存在包冲突时，用户可以使用 Maven 插件将若干冲突包打包成 Plugin，运行时由独立的 PluginClassLoader 加载，从而解决包冲突。

将复杂项目拆分到多个工程的主要动机包括避免VCS提交冲突、规避技术栈导致的依赖冲突。SOFA支持模块化，可以将多个业务模块打包为Biz，在运行时合并部署到单个JVM，各模块基于同一的API进行交互。

这种模式下，Biz之间的依赖可以通过Maven管理，当应用打为扁平化JAR时其依赖的Biz可以合并进来。每个Biz使用独立的BizClassLoader加载，Biz之间通过JVM服务（SofaService/SofaRefernece）进行交互。

这种模式下，Biz可以在运行时，通过API或者配置中心（ZooKeeper）来动态部署/卸载。

这里需要提到主应用（Master Biz）的概念，其实不管静态/动态合并，此概念都是存在的。如果Ark包打了单个Biz则它就是主应用，如果打了多个Biz包则需要配置指定主应用。主应用不得卸载。

通常情况下，各模块的实现放在动态Biz中，供主应用调用。主应用通过两种方式部署/卸载动态Biz：

1. 调用SOFAArk的API
2. 使用SOFAArk的Config插件，对接ZooKeeper配置中心。此插件会解析配置并控制动态Biz的部署/卸载

Ark Plugin可以导入、导出类或资源：

1. 导入类：插件启动时，优先委托给导出该类的插件负责加载，如果加载不到，才会尝试从本插件内部加载

2. 导出类：其他插件如果导入了该类，优先从本插件加载

3. 导入资源：插件在查找资源时，优先委托给导出该资源的插件负责加载，如果加载不到，才会尝试从本插件内部加载

4. 导出资源：其他插件如果导入了该资源，优先从本插件加载

SOFAArk的代码库提供了一个样例项目。其结构如下：

├── sample-ark-plugin
│   ├── common    # 此模块包含了插件导出类
│   ├── plugin    # 包含插件服务的实现、PluginActivator接口实现
│   ├── pom.xml

plugin项目的POM中包含如下配置：

<build>
    <plugins>
        <plugin>
            <groupId>com.alipay.sofa</groupId>
            <artifactId>sofa-ark-plugin-maven-plugin</artifactId>
            <version>${project.version}</version>
            <executions>
                <execution>
                    <id>default-cli</id>
                    <goals>
                        <goal>ark-plugin</goal>
                    </goals>

                    <configuration>

                        <!-- Ark 容器启动插件的入口类，最多只能配置一个。一般在此执行初始化操作，比如发布插件服务 -->
                        <activator>com.alipay.sofa.ark.sample.activator.SamplePluginActivator</activator>

                        <!-- 导出的包、类、资源列表 -->
                        <exported>
                            <packages>
                                <package>com.alipay.sofa.ark.sample.common</package>
                            </packages>
                            <classes>
                                <class>com.alipay.sofa.ark.sample.facade.SamplePluginService</class>
                            </classes>
                        </exported>

                        <!-- 打包后的插件的存放位置，默认${project.build.directory} -->
                        <outputDirectory>../target</outputDirectory>

                    </configuration>
                </execution>

            </executions>
        </plugin>
    </plugins>
</build>

此配置声明了当前插件的一切必要信息。

SamplePluginActivator负责在插件启动时，发布JVM服务：

package com.alipay.sofa.ark.sample.activator;

import com.alipay.sofa.ark.exception.ArkRuntimeException;
import com.alipay.sofa.ark.sample.facade.SamplePluginService;
import com.alipay.sofa.ark.sample.impl.SamplePluginServiceImpl;
import com.alipay.sofa.ark.spi.model.PluginContext;
import com.alipay.sofa.ark.spi.service.PluginActivator;

public class SamplePluginActivator implements PluginActivator {

    // 插件启动时的回调
    public void start(PluginContext context) throws ArkRuntimeException {
        // 通过插件上下文发布服务
        context.publishService(SamplePluginService.class, new SamplePluginServiceImpl());
    }

    // 插件停止时的回调
    public void stop(PluginContext context) throws ArkRuntimeException {
        System.out.println("stopping in ark plugin activator");
    }

}

除了发布服务之外，你还可以引用其他插件或Ark容器发布的服务：

public class SamplePluginServiceImpl implements SamplePluginService {

    // 引用Ark容器发布的，事件管理服务
    @ArkInject
    private EventAdminService eventAdminService;

    public String service() {
        return "I'm a sample plugin service published by ark-plugin";
    }

    public void sendEvent(ArkEvent arkEvent) {
        eventAdminService.sendEvent(arkEvent);
    }
}

使用sofa-ark-maven-plugin也可以把普通的Spring Boot项目打为Ark包：

<build>
    <plugins>
        <plugin>
            <groupId>com.alipay.sofa</groupId>
            <artifactId>sofa-ark-maven-plugin</artifactId>
            <executions>
                <execution>
                    <id>default-cli</id>
                    
                    <!-- 此目标生成可执行Ark包 -->
                    <goals>
                        <goal>repackage</goal>
                    </goals>
                    
                    <configuration>
                        <!-- 可执行Ark包的输出目录 -->
                        <outputDirectory>./target</outputDirectory>
                        <!-- 可以指定Ark包的Maven坐标的classifier字段 -->
                        <arkClassifier>executable-ark</arkClassifier>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

添加以下依赖即可：

<dependency>
    <groupId>com.alipay.sofa</groupId>
    <artifactId>sofa-ark-springboot-starter</artifactId>
    <version>${sofa.ark.version}</version>
</dependency>

需要添加依赖：

<dependency>
    <groupId>com.alipay.sofa</groupId>
    <artifactId>sofa-ark-support-starter</artifactId>
    <version>${sofa.ark.version}</version>
</dependency>

并且在入口点方法中启动Ark容器：

public class Application{
    public static void main(String[] args) { 
        SofaArkBootstrap.launch(args);
    }
}

SOFAArk提供了org.junit.runner.Runner的实现类ArkJUnit4Runner，用于集成JUnit4单元测试框架：

@RunWith(ArkJUnit4Runner.class)
public class JUnitTest {

    @Test
    public void test() {
        Assert.assertTrue(true);
    }

上面的用例会在Ark容器之上运行。 

SOFAArk提供了org.junit.runner.Runner的实现类ArkBootRunner，用于在Spring Boot下进行集成测试：

@RunWith(ArkBootRunner.class)
@SpringBootTest(classes = SpringbootDemoApplication.class)
public class IntegrationTest {

    // 可以注入依赖
    @Autowired
    private SampleService sampleService;

    @Test
    public void test() {
        sampleService.service();
    }

} 

这是一个Java的RPC框架，提供了负载均衡，流量转发，链路追踪，链路数据透传，故障剔除等特性，兼容 bolt，RESTful，dubbo，H2C协议。

SOFARPC的基本工作原理和Dubbo类似：

1. 如果当前应用需要发布RPC服务，那么SOFARPC会将服务注册到服务注册中心
2. 如果当前应用需要调用RPC服务，那么SOFARPC会到注册中心订阅相关服务的元数据。注册中心会即时的推送元数据更新，例如服务的端点信息

需要加入依赖：

<dependency>
    <groupId>com.alipay.sofa</groupId>
    <artifactId>sofa-rpc-all</artifactId>
    <version>5.6.0-SNAPSHOT</version>
</dependency>

需要发布的服务接口： 

public interface HelloService {
    String sayHello(String string);
}

接口实现HelloServiceImpl这里略去。

SOFARPC服务器：

public class QuickStartServer {

    public static void main(String[] args) {

        ServerConfig serverConfig = new ServerConfig()
            .setProtocol("bolt") // 设置一个协议，默认bolt
            .setPort(12200) // 设置一个端口，默认12200
            .setDaemon(false); // 非守护线程方式运行

        // 使用ZK作为注册表
        RegistryConfig registryConfig = new RegistryConfig()
            .setProtocol("zookeeper")
            .setAddress("127.0.0.1:2181");

        ProviderConfig providerConfig = new ProviderConfig()
            .setInterfaceId(HelloService.class.getName()) // 指定接口
            .setRef(new HelloServiceImpl()) // 指定实现
            .setServer(serverConfig)  // 指定服务端
            .setRegistry(registryConfig);  // 服务注册表

        providerConfig.export(); // 发布服务
    }
}

SOFARPC客户端： 

public class QuickStartClient {

    private final static Logger LOGGER = LoggerFactory.getLogger(QuickStartClient.class);

    public static void main(String[] args) {

        ConsumerConfig consumerConfig = new ConsumerConfig()
            .setInterfaceId(HelloService.class.getName()) // 指定接口
            .setProtocol("bolt") // 指定协议
            .setDirectUrl("bolt://127.0.0.1:12200") // 指定服务端的直连地址
            .setConnectTimeout(10 * 1000);

        HelloService helloService = consumerConfig.refer();

        helloService.sayHello("world");
    }
}

SpringBoot应用的名字必须配置：

spring.application.name=test 

你需要为SOFABoot工程引入SOFARPC的starter：

<dependency>
     <groupId>com.alipay.sofa</groupId>
     <artifactId>rpc-sofa-boot-starter</artifactId>
</dependency>

你可以以POJO的形式定义服务接口和它的实现。

发布服务时，可以使用下面的XML配置：

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:sofa="http://sofastack.io/schema/sofaboot"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
            http://sofastack.io/schema/sofaboot   http://sofastack.io/schema/sofaboot.xsd"
       default-autowire="byName">
    
    <!-- 服务的Bean定义 -->
    <bean id="personServiceImpl" class="com.alipay.sofa.boot.examples.demo.rpc.bean.PersonServiceImpl"/>

    <!-- 发布SOFARPC服务 -->
    <sofa:service ref="personServiceImpl" interface="com.alipay.sofa.boot.examples.demo.rpc.bean.PersonService">
        <!-- 绑定的通信协议  -->
        <sofa:binding.bolt>
            <sofa:global-attrs timeout="3000" address-wait-time="2000"/>  <!-- 调用超时；地址等待时间 -->
            <sofa:route target-url="127.0.0.1:22000"/>  <!-- 直连到提供者，不走负载均衡 -->
            <sofa:method name="sayName" timeout="3000"/> <!-- 方法级别超时配置 -->
        </sofa:binding.bolt>
        <sofa:binding.rest/>
    </sofa:service>

    <!-- 订阅SOFARPC服务 -->
    <sofa:reference id="personReferenceBolt" interface="com.alipay.sofa.boot.examples.demo.rpc.bean.PersonService">
        <sofa:binding.bolt/>
    </sofa:reference>

    <sofa:reference id="personReferenceRest" interface="com.alipay.sofa.boot.examples.demo.rpc.bean.PersonService">
        <sofa:binding.rest/>
    </sofa:reference>

</beans>

// 发布服务
@SofaService(interfaceType = AnnotationService.class, 
               bindings = { 
                 @SofaServiceBinding(bindingType = "bolt"),
                 @SofaServiceBinding(bindingType = "bolt") 
               })
@Component
public class AnnotationServiceImpl implements AnnotationService {
    @Override
    public String sayAnnotation(String stirng) {
        return stirng;
    }
}

// 订阅服务
@Component
public class AnnotationClientImpl {

    @SofaReference(interfaceType = AnnotationService.class, 
                     binding = @SofaReferenceBinding(bindingType = "bolt"))
    private AnnotationService annotationService;

    public String sayClientAnnotation(String str) {

        String result = annotationService.sayAnnotation(str);

        return result;
    }
}

SOFARPC支持过滤器。实现过滤器非常简单：

public class PersonServiceFilter extends Filter {
    @Override
    public SofaResponse invoke(FilterInvoker invoker, SofaRequest request) throws SofaRpcException {
        // 前置钩子
        System.out.println("PersonFilter before");
        try {
            // 执行RPC调用
            return invoker.invoke(request);
        } finally {
        // 后置钩子
            System.out.println("PersonFilter after");
        }
    }
}

过滤器需要配置才能生效。

Bolt是一个高性能的TCP协议，其性能比HTTP好。

Bolt支持同步、异步、回调、单向等多种调用类型：

<sofa:reference interface="com.example.demo.SampleService" id="sampleService">
    <sofa:binding.bolt>
        <sofa:global-attrs type="future"/>
    </sofa:binding.bolt>
</sofa:reference>

@SofaReference(binding = @SofaReferenceBinding(bindingType = "bolt", invokeType = "future"))
private SampleService sampleService;

BoltBindingParam boltBindingParam = new BoltBindingParam();
boltBindingParam.setType("future");

ConsumerConfig consumerConfig = new ConsumerConfig()
    .setInterfaceId(SampleService.class.getName())
    .setRegistry(registryConfig)
    .setProtocol("bolt")
    .setInvokeType("future");

// 第一个参数为超时，第二个参数提示是否删除线程上下文中的调用结果缓存
String result = (String)SofaResponseFuture.getResponse(0, true);

Future future = SofaResponseFuture.getFuture(true);

/**
 * 面向用户的Rpc请求结果监听器
 *
 */
public interface SofaResponseCallback {
    /**
     * SOFA RPC 会在调用成功的响应到达后回调此方法
     *
     * @param appResponse 响应对象
     * @param methodName 被调用的方法
     * @param request 请求对象
     */
    void onAppResponse(Object appResponse, String methodName, RequestBase request);

    /**
     * SOFA RPC 会在服务器端异常时回调此方法
     */
    void onAppException(Throwable throwable, String methodName, RequestBase request);

    /**
     * SOFA RPC 会在框架异常时回调此方法
     *
     */
    void onSofaException(SofaRpcException sofaException, String methodName, RequestBase request);
}

<!-- 回调Bean -->
<bean id="sampleCallback" class="com.example.demo.SampleCallback"/>
<sofa:reference interface="com.example.demo.SampleService" id="sampleService">
    <sofa:binding.bolt>
        <!--                               引用回调Bean                 -->
        <sofa:global-attrs type="callback" callback-ref="sampleCallback"/>
    </sofa:binding.bolt>
</sofa:reference>

@SofaReference(binding = @SofaReferenceBinding(bindingType = "bolt",
            invokeType = "callback",
            callbackRef = "sampleCallback"))
private SampleService sampleService;

BoltBindingParam boltBindingParam = new BoltBindingParam();
boltBindingParam.setType("callback");
boltBindingParam.setCallbackClass("com.example.demo.SampleCallback");

ConsumerConfig consumerConfig = new ConsumerConfig()
    .setInterfaceId(SampleService.class.getName())
    .setRegistry(registryConfig)
    .setProtocol("bolt")
    .setInvokeType("callback")
    .setOnReturn(new SampleCallback());

RpcInvokeContext.getContext().setResponseCallback(new SampleCallback());

使用Bolt协议时默认的超时是3s。你可以在多个级别设置超时。

在服务级别设置：

<sofa:reference interface="com.example.demo.SampleService" id="sampleService">
    <sofa:binding.bolt>
        <sofa:global-attrs timeout="2000"/>
    </sofa:binding.bolt>
</sofa:reference>

@SofaReference(binding = @SofaReferenceBinding(bindingType = "bolt", timeout = 2000)) 
private SampleService sampleService;

在方法级别设置： 

<sofa:reference interface="com.example.demo.SampleService" id="sampleService">
    <sofa:binding.bolt>
        <sofa:method name="hello" timeout="2000"/>
    </sofa:binding.bolt>
</sofa:reference>

SOFARPC允许消费者在不知道服务接口的情况下发起调用。前提条件是：

1. 使用Bolt作为通信协议
2. 使用Hessian 2作为串行化协议 

SOFARPC支持Hessian 2、protobuf两个串行化协议，默认使用前者。如果要修改，参考：

sofa:service ref="sampleService" interface="com.alipay.sofarpc.demo.SampleService">
    <sofa:binding.bolt>
        <sofa:global-attrs serialize-type="protobuf"/>
    </sofa:binding.bolt>
</sofa:service>
<!-- 提供者/消费者都需要设置 -->
<sofa:reference interface="com.alipay.sofarpc.demo.SampleService" id="sampleServiceRef" jvm-first="false">
    <sofa:binding.bolt>
        <sofa:global-attrs serialize-type="protobuf"/>
    </sofa:binding.bolt>
</sofa:reference>

<bean id="helloService" class="com.alipay.sofa.rpc.quickstart.HelloService"/>

<!-- 自定义一个线程池 -->
<bean id="customExecutor" class="com.alipay.sofa.rpc.server.UserThreadPool" init-method="init">
    <property name="corePoolSize" value="10" />
    <property name="maximumPoolSize" value="10" />
    <property name="queueSize" value="0" />
</bean>

<sofa:service ref="helloService" interface="XXXService">
    <sofa:binding.bolt>
        <!-- 引用线程池 -->
        <sofa:global-attrs thread-pool-ref="customExecutor"/>
    </sofa:binding.bolt>
</sofa:service>

或者，使用注解方式：

@SofaService(bindings = {@SofaServiceBinding(bindingType = "bolt", userThreadPool = "customThreadPool")})
public class SampleServiceImpl implements SampleService {
}

SOFARPC支持RESTful协议，使用此协议时，你需要为服务接口添加JAX-RS注解：

@Path("sample")
public interface SampleService {
    @GET
    @Path("hello")
    String hello();
}

发布服务的方式如下：

@Service
//                                           设置绑定类型
@SofaService(bindings = {@SofaServiceBinding(bindingType = "rest")})
public class RestfulSampleServiceImpl implements SampleService {
    @Override
    public String hello() {
        return "Hello";
    }
}

这种服务直接可以通过浏览器访问： http://localhost:8341/sample/hello

在SOFARPC客户端，消费服务的方式如下：

@SofaReference(binding = @SofaReferenceBinding(bindingType = "rest"))
private SampleService sampleService;

SOFARPC支持多种注册中心。当前bolt、rest、duboo传输协议均支持ZooKeeper作为注册中心，bolt、rest还支持本地文件系统作为注册中心（主要用于测试）。

当前SOFARPC（SOFARPC: 5.5.2, SOFABoot: 2.6.3）已经支持SOFARegistry注册中心：

com.alipay.sofa.rpc.registry.address=sofa://127.0.0.1:9603

要使用此注册中心，配置：

com.alipay.sofa.rpc.registry.address=zookeeper://127.0.0.1:2181
# 指定身份验证信息
com.alipay.sofa.rpc.registry.address=zookeeper://xxx:2181?file=/home/admin/registry&scheme=digest&addAuth=sofazk:rpc1

要使用此注册中心，配置：

com.alipay.sofa.rpc.registry.address=local:///home/admin/registry/localRegistry.reg

服务消费者可以直接指定服务提供者的地址： 

ConsumerConfig consumer = new ConsumerConfig()        
            .setInterfaceId(HelloService.class.getName())        
            .setRegistry(registryConfig)        
            .setDirectUrl("bolt://127.0.0.1:12201");

<sofa:reference interface="com.alipay.sample.HelloService" id="helloService">
    <sofa:binding.bolt>
        <sofa:route target-url="127.0.0.1:12200"/>
    </sofa:binding.bolt>
</sofa:reference>

@SofaReference(binding = @SofaReferenceBinding(bindingType = "bolt", directUrl = "127.0.0.1:12220"))
private SampleService sampleService;

而不经过SOFARPC的负载均衡系统。

目前支持的负载均衡算法：

配置负载均衡算法的方式如下： 

<sofa:reference interface="com.example.demo.SampleService" id="sampleService">
    <sofa:binding.bolt>
        <sofa:global-attrs loadBalancer="roundRobin"/>
    </sofa:binding.bolt>
</sofa:reference>

重试的配置方式如下：

<sofa:reference jvm-first="false" id="retriesServiceReferenceBolt" interface="com.alipay.sofa.rpc.samples.retries.RetriesService">
   <sofa:binding.bolt>
     <sofa:global-attrs retries="2"/>
   </sofa:binding.bolt>
</sofa:reference>

@SofaReference(binding = @SofaReferenceBinding(bindingType = "bolt", retries = 2))
private SampleService sampleService;

SOFARPC目前支持以下Tracer：

1. SOFATracer，内置集成
2. Skywalking

如果要禁用分布式追踪特性，可以配置：

com.alipay.sofa.rpc.defaultTracer=

SOFARPC支持内置的容错机制，还可以和Hystrix进行集成。

运行机制：

1. 单机故障剔除会统计一个时间窗口内的调用次数和异常次数，并计算每个服务对应ip的异常率和该服务的平均异常率
2. 当达到ip异常率大于服务平均异常率到一定比例时，会对服务+ip的维度进行权重降级
3. 如果该服务+ip维度的权重并没有降为0，那么当该服务+ip维度的调用情况正常时，则会对其进行权重恢复
4. 整个计算和调控过程异步进行，不会阻塞调用

使用这种单机故障剔除，需要进行如下配置：

FaultToleranceConfig faultToleranceConfig = new FaultToleranceConfig();
        faultToleranceConfig.setRegulationEffective(true);
        faultToleranceConfig.setDegradeEffective(true);
        // 时间窗口 20s
        faultToleranceConfig.setTimeWindow(20);
        // 如果被判定为故障，则权重掉1/2
        faultToleranceConfig.setWeightDegradeRate(0.5);

FaultToleranceConfigManager.putAppConfig("appName", faultToleranceConfig);

基于Hystrix的熔断能力，目前还不健全。要使用此特性，添加Hystrix的依赖：

<dependency>
        <groupId>com.netflix.hystrix</groupId>
        <artifactId>hystrix-core</artifactId>
        <version>1.5.12</version>
</dependency>

然后显式启用Hystrix（导致Hystrix过滤器被加载）： 

// 全局开启
RpcConfigs.putValue(HystrixConstants.SOFA_HYSTRIX_ENABLED, true);

// 对特定 Consumer 开启
ConsumerConfig consumerConfig = new ConsumerConfig()
        .setInterfaceId(HelloService.class.getName())
        .setParameter(HystrixConstants.SOFA_HYSTRIX_ENABLED, String.valueOf(true));

FallbackFactory接口用于注入Hystrix的Fallback，当Hystrix遇到异常、超时、线程池拒绝、熔断等情况时，指定执行此Fallback（降级）逻辑：

// 可以直接使用默认的 FallbackFactory 直接注入 Fallback 实现
SofaHystrixConfig.registerFallback(consumerConfig, new HelloServiceFallback());

// 也可以自定义 FallbackFactory 直接注入 FallbackFactory
SofaHystrixConfig.registerFallbackFactory(consumerConfig, new HelloServiceFallbackFactory());

SOFARPC注册了JVM的ShutdownHook，用于实现优雅的资源清理。 

这是一个服务注册中心，和ZooKeeper、Etcd等项目对比有自己的特点：

1. 高度可扩容：数据分片存储，理论上可以支持无限水平扩容
2. 低延迟：基于SOFABolt框架，实现TCP长连接下的变更推送。主流注册中心都是这种架构
3. 高可用：在CAP中保证AP，放弃严格一致性，尽可能在网络分区时保证可用性。这和ZooKeeper、Etcd不同。需要注意的是，Envoy xDS协议也是最终一致性的

在架构上，SOFARegistry引入4种角色：

1. Client：通过客户端JAR包接入注册中心
2. SessionServer：直接处理Client的服务发布/订阅请求，可以无限扩容。客户端仅和SessionServer交互
3. DataServer：负责存储服务的元数据，使用一致性哈希分片存储，支持多副本，可以无限扩容
4. MetaServer：维护SessionServer、DataServer集群的一致性列表，在节点发生变更时发出通知

支持独立部署、集成部署方式。后者比较简单，单节点部署，可以用于测试。

wget https://github.com/alipay/sofa-registry/releases/download/v5.2.0/registry-integration-5.2.0.tar.gz
mkdir registry-integration
tar -zxvf registry-integration-5.2.0.tar.gz -C registry-integration
cd registry-integration

# 启动脚本位于
bin/startup.sh

启动服务器后，执行下面的命令查看各服务器端角色的健康状况：

# 查看meta角色的健康检测接口：
curl http://localhost:9615/health/check
# {"success":true,"message":"... raftStatus:Leader"}

# 查看data角色的健康检测接口：
curl http://localhost:9622/health/check
# {"success":true,"message":"... status:WORKING"}

# 查看session角色的健康检测接口：
curl http://localhost:9603/health/check
# {"success":true,"message":"..."}

引入依赖：

<dependency>    
    <groupId>com.alipay.sofa</groupId>
    <artifactId>registry-client-all</artifactId>
</dependency>

下面的代码演示了如何发布数据到SOFARegistry上： 

// 构建客户端实例
RegistryClientConfig config = DefaultRegistryClientConfigBuilder.start()
    // 服务器连接地址，任意一个Session节点都可以
    .setRegistryEndpoint("127.0.0.1").setRegistryEndpointPort(9603).build();
DefaultRegistryClient registryClient = new DefaultRegistryClient(config);
registryClient.init();

// 构造发布者注册表
// dataId是此客户端发布的服务的唯一标识
String dataId = "com.alipay.test.demo.service:1.0@DEFAULT";
PublisherRegistration registration = new PublisherRegistration(dataId);

// 将注册表注册进客户端并发布数据
registryClient.register(registration, "10.10.1.1:12200?xx=yy");

下面的代码演示了如何从SOFARegistry订阅数据：

// 构建客户端实例
RegistryClientConfig config = DefaultRegistryClientConfigBuilder.start()
    .setRegistryEndpoint("127.0.0.1").setRegistryEndpointPort(9603).build();
DefaultRegistryClient registryClient = new DefaultRegistryClient(config);
registryClient.init();

// 创建 SubscriberDataObserver 
SubscriberDataObserver subscriberDataObserver = new SubscriberDataObserver() {
  	public void handleData(String dataId, UserData userData) {
                // 在这里处理接收到的数据
    		System.out.println("receive data success, dataId: " + dataId + ", data: " + userData);
  	}
};

// 构造订阅者注册表
String dataId = "com.alipay.test.demo.service:1.0@DEFAULT";
SubscriberRegistration registration = new SubscriberRegistration(dataId, subscriberDataObserver);
// 订阅的范围：zone, dataCenter, global
registration.setScopeEnum(ScopeEnum.global);

// 将注册表注册进客户端并订阅数据，订阅到的数据会以回调的方式通知 SubscriberDataObserver
registryClient.register(registration);

有数据更新后，服务器会推送并回调，你需要在回调中处理 UserData：

public interface UserData {
    // 以Zone分组的数据
    Map<String, List> getZoneData();
    // 当前Zone
    String getLocalZone();
}

遵循OpenTracing规范的Tracer。可以将将一个Trace的链路信息打印到日志或发送给Zipkin进行展示。 特性包括：

1. 基于Disruptor实现高性能的Trace日志落盘。支持两种打印类型：
   1. 摘要日志：每一次调用均会落地磁盘的日志
   2. 统计日志：每隔一定时间间隔进行统计输出的日志
2. 支持日志自清除和轮换
3. 集成SLF4J的MDC，修改日志配置即可输出当前Trace上下文的TraceId 和 SpanId
4. 已开发多种开源项目的埋点：
   1. Spring MVC
   2. 基于标准JDBC接口的数据库连接池，包括DBCP、Druid、c3p0、tomcat、HikariCP、BoneCP
   3. HttpClient
   4. RestTemplate
   5. OkHttp
   6. Dubbo
   7. OpenFeign
   8. Redis、消息中间件的埋点仍然在开发中 

# 启动Trace的那个服务器的IP地址，十六进制
#        Trace产生的时间戳
#                      自增序列
#                           当前进程ID   
0ad1348f 1403169275002 1003 56696

SLF4J 提供了 MDC （Mapped Diagnostic Contexts）功能，支持用户定义和修改日志的输出格式以及内容。SOFATracer支持基于MDC来输出当前Trace上下文的TraceId、SpanId。

需要引入SLF4J的API，以及日志实现包的Maven依赖：

<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
</dependency>

<!-- Logback -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-logging</artifactId>
</dependency>

<!-- Log4j2 -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-log4j2</artifactId>
    <!--SOFABoot没有管控log4j2 版本 -->
    <version>1.4.2.RELEASE</version>
</dependency>

配置日志输出的PatternLayout：

<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %5p  [%X{SOFA-TraceId}, %X{SOFA-SpanId}] -- %m%n</pattern>

%X表示，在调用appender输出日志时，将此占位符替换为当前线程上下文（MDC）中的SOFA-TraceId、SOFA-SpanId变量。

如果用户启动新线程来处理业务，则基于线程本地变量传递的Trace信息就丢失了。为了将SOFATracer日志上下文从父线程传递到子线程，可以考虑用SofaTracerRunnable：

//                         使用SOFATracer提供的包装器
Thread thread = new Thread(new SofaTracerRunnable(new Runnable() {
            @Override
            public void run() {
                // 异步业务逻辑
            }
        }));
thread.start();

基于java.util.concurrent.Callable发动新线程时类似：

ExecutorService executor = Executors.newCachedThreadPool();
//                                                            使用SOFATracer提供的包装器
SofaTracerCallable<Object> sofaTracerSpanSofaTracerCallable = new SofaTracerCallable<Object>(new Callable<Object>() {
    @Override
    public Object call() throws Exception {
        // 异步业务逻辑
        return ...;
    }
});
Future<Object> futureResult = executor.submit(sofaTracerSpanSofaTracerCallable);
// ...
Object objectReturn = futureResult.get();

SOFATracer支持两种采样模式：

1.  基于BitSet实现的固定采样率的采样模式
2. 用户自定义实现采样的采样模式

进行以下配置即可：

# 采样率0-100之间
com.alipay.sofa.tracer.samplerPercentage=100
# 采样模式类型名称
com.alipay.sofa.tracer.samplerName=PercentageBasedSampler

此项目解决的是监控领域的问题，提供指标的埋点、收集、加工、存储、查询等服务，分为客户端、服务器两个部分。

此项目已经快一年没有更新。

扩展了Istio项目，并做了以下改进：

1. 将数据平面的Envoy替换为SOFAMosn
2. 下沉Mixer的功能到数据平面，提升性能
3. 扩展Pilot，支持更多的服务发现机制（服务注册表），包括SOFARPC、Dubbo

SOFARPC、Dubbo之类的入侵式框架可以在Istio中运行，但是无法对其进行任何管理、监控。SOFAMesh解决了这些痛点。

MOSN（Modular Observable Smart Network，模块化可观察智能网络…）是Envoy的替代品，其出现的主要原因不是Envoy不行，而是阿里系不愿引入C++技术栈。

MOSN增加了对SOFARPC、Dubbo协议的支持，后者仍然在开发中。

The post SOFAStack学习笔记 appeared first on 绿色记忆.

Bazel是Google开源的，类似于Make、Maven或Gradle的构建和测试工具。它使用可读性强的、高层次的构建语言，支持多种编程语言，以及为多种平台进行交叉编译。

Bazel的优势：

1. 高层次的构建语言：更加简单，Bazel抽象出库、二进制、脚本、数据集等概念，不需要编写调用编译器或链接器的脚本
2. 快而可靠：能够缓存所有已经完成的工作步骤，并且跟踪文件内容、构建命令的变动情况，避免重复构建。此外Bazel还支持高度并行构建、增量构建
3. 多平台支持：可以在Linux/macOS/Windows上运行，可以构建在桌面/服务器/移动设备上运行的应用程序
4. 可扩容性：处理10万以上源码文件时仍然能保持速度
5. 可扩展性：支持Android、C/C++、Java、Objective-C、Protocol Buffer、Python…还支持扩展以支持其它语言

当运行构建或者测试时，Bazel会：

1. 加载和目标相关的BUILD文件
2. 分析输入及其依赖，应用指定的构建规则，产生一个Action图。这个图表示需要构建的目标、目标之间的关系，以及为了构建目标需要执行的动作。Bazel依据此图来跟踪文件变动，并确定哪些目标需要重新构建
3. 针对输入执行构建动作，直到最终的构建输出产生出来

当你需要构建或者测试一个项目时，通常执行以下步骤：

1. 下载并安装Bazel
2. 创建一个工作空间。Bazel从此工作空间寻找构建输入和BUILD文件，同时也将构建输出存放在（指向）工作空间（的符号链接中）
3. 编写BUILD文件，以及可选的WORKSPACE文件，告知Bazel需要构建什么，如何构建。此文件基于Starlark这种DSL
4. 从命令行调用Bazel命令，构建、测试或者运行项目

工作空间是一个目录，它包含：

1. 构建目标所需要的源码文件，以及相应的BUILD文件
2. 指向构建结果的符号链接
3. WORKSPACE文件，可以为空，可以包含对外部依赖的引用

包是工作空间中主要的代码组织单元，其中包含一系列相关的文件（主要是代码）以及描述这些文件之间关系的BUILD文件

包是工作空间的子目录，它的根目录必须包含文件BUILD.bazel或BUILD。除了那些具有BUILD文件的子目录——子包——以外，其它子目录属于包的一部分

包是一个容器，它的元素定义在BUILD文件中，包括：

1. 规则（Rule），指定输入集和输出集之间的关系，声明从输入产生输出的必要步骤。一个规则的输出可以是另外一个规则的输入
2. 文件（File），可以分为两类：
   1. 源文件
   2. 自动生成的文件（Derived files），由构建工具依据规则生成
3. 包组：一组包，包组用于限制特定规则的可见性。包组由函数package_group定义，参数是包的列表和包组名称。你可以在规则的visibility属性中引用包组，声明那些包组可以引用当前包中的规则

任何包生成的文件都属于当前包，不能为其它包生成文件。但是可以从其它包中读取输入

引用一个目标时需要使用“标签”。标签的规范化表示：

@project//my/app/main:app_binary

//my/app
//my/app:app

这两者是等价的。在BUILD文件中，引用当前包中目标时，包名部分可以省略，因此下面四种写法都可以等价：

# 当前包为my/app
//my/app:app
//my/app
:app
app

在BUILD文件中，引用当前包中定义的规则时，冒号不能省略。引用当前包中文件时，冒号可以省略。 例如：

generate.cc

但是，从其它包引用时、从命令行引用时，都必须使用完整的标签：

//my/app:generate.cc

@project这一部分通常不需要使用，引用外部存储库中的目标时，project填写外部存储库的名字。

规则指定输入和输出之间的关系，并且说明产生输出的步骤。

规则有很多类型。每个规则都具有一个名称属性，此名称亦即目标名称。对于某些规则，此名称就是产生的输出的文件名。

在BUILD中声明规则的语法时：

规则类型(
    name = "...",
    其它属性 = ...
)

BUILD文件定义了包的所有元数据。其中的语句被从上而下的逐条解释，某些语句的顺序很重要， 例如变量必须先定义后使用，但是规则声明的顺序无所谓。

BUILD文件仅能包含ASCII字符，且不得声明函数、使用for/if语句，你可以在Bazel扩展——扩展名为.bzl的文件中声明函数、控制结构。并在BUILD文件中用load语句加载Bazel扩展：

load("//foo/bar:file.bzl", "some_library")

上面的语句加载foo/bar/file.bzl并添加其中定义的符号some_libraray到当前环境中，load语句可以用来加载规则、函数、常量（字符串、列表等）。

load语句必须出现在顶级作用域，不能出现在函数中。第一个参数说明扩展的位置，你可以为导入的符号设置别名。

规则的类型，一般以编程语言为前缀，例如cc，java，后缀通常有：

1. *_binary 用于构建目标语言的可执行文件
2. *_test 用于自动化测试，其目标是可执行文件，如果测试通过应该退出0
3. *_library 用于构建目标语言的库 

目标A依赖B，就意味着A在构建或执行期间需要B。所有目标的依赖关系构成非环有向图（DAG）称为依赖图。

距离为1的依赖称为直接依赖，大于1的依赖则称为传递性依赖。

依赖分为以下几种：

1. srcs依赖：直接被当前规则消费的文件
2. deps依赖：独立编译的模块，为当前规则提供头文件、符号、库、数据
3. data依赖：不属于源码，不影响目标如何构建，但是目标在运行时可能依赖之

参考下面的步骤安装Bazel：

echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add -

sudo apt-get update && sudo apt-get install bazel

可以用如下命令升级到最新版本的Bazel：

sudo apt-get install --only-upgrade bazel

这是基于Go语言编写的Bazel启动器，它会为你的工作区下载最适合的Bazel，并且透明的将命令转发给该Bazel。

由于Bazellisk提供了和Bazel一样的接口，因此通常直接将其命名为bazel：

sudo wget -O /usr/local/bin/bazel https://github.com/bazelbuild/bazelisk/releases/download/v0.0.8/bazelisk-linux-amd64
sudo chmod +x /usr/local/bin/bazel 

执行下面的命令下载示例项目：

git clone https://github.com/bazelbuild/examples/

你可以看到stage1、stage2、stage3这几个WORKSPACE：

examples
└── cpp-tutorial
    ├──stage1
    │  ├── main
    │  │   ├── BUILD
    │  │   └── hello-world.cc
    │  └── WORKSPACE
    ├──stage2
    │  ├── main
    │  │   ├── BUILD
    │  │   ├── hello-world.cc
    │  │   ├── hello-greet.cc
    │  │   └── hello-greet.h
    │  └── WORKSPACE
    └──stage3
       ├── main
       │   ├── BUILD
       │   ├── hello-world.cc
       │   ├── hello-greet.cc
       │   └── hello-greet.h
       ├── lib
       │   ├── BUILD
       │   ├── hello-time.cc
       │   └── hello-time.h
       └── WORKSPACE

本节后续内容会依次使用到这三个WORKSPACE。

第一步是创建工作空间。工作空间中包含以下特殊文件：

1. WORKSPACE，此文件位于根目录中，将当前目录定义为Bazel工作空间
2. BUILD，告诉Bazel项目的不同部分如何构建。工作空间中包含BUILD文件的目录称为包

当Bazel构建项目时，所有的输入和依赖都必须位于工作空间中。除非被链接，不同工作空间的文件相互独立没有关系。

每个BUILD文件包含若干Bazel指令，其中最重要的指令类型是构建规则（Build Rule），构建规则说明如何产生期望的输出——例如可执行文件或库。 BUILD中的每个构建规则也称为目标（Target），目标指向若干源文件和依赖，也可以指向其它目标。

下面是stage1的BUILD文件：

cc_binary(
    name = "hello-world",
    srcs = ["hello-world.cc"],
)

这里定义了一个名为hello-world的目标，它使用了内置的cc_binary规则。该规则告诉Bazel，从源码hello-world.cc构建一个自包含的可执行文件。

执行下面的命令可以触发构建：

#   //main: BUILD文件相对于工作空间的位置
#          hello-world 是BUILD文件中定义的目标
bazel build //main:hello-world

构建完成后，工作空间根目录会出现bazel-bin等目录，它们都是指向$HOME/.cache/bazel某个后代目录的符号链接。执行：

bazel-bin/main/hello-world

可以运行构建好的二进制文件。

Bazel会根据BUILD中的声明产生一张依赖图，并根据这个依赖图实现精确的增量构建。

要查看依赖图，先安装：

sudo apt install graphviz xdot

然后执行：

bazel query --nohost_deps --noimplicit_deps 'deps(//main:hello-world)' --output graph | xdot

大型项目通常会划分为多个包、多个目标，以实现更快的增量构建、并行构建。工作空间stage2包含单个包、两个目标：

# 首先构建hello-greet库，cc_library是内建规则
cc_library(
    name = "hello-greet",
    srcs = ["hello-greet.cc"],
    # 头文件
    hdrs = ["hello-greet.h"],
)

# 然后构建hello-world二进制文件
cc_binary(
    name = "hello-world",
    srcs = ["hello-world.cc"],
    deps = [
        # 提示Bazel，需要hello-greet才能构建当前目标
        # 依赖当前包中的hello-greet目标
        ":hello-greet",
    ],
)

工作空间stage3更进一步的划分出新的包，提供打印时间的功能：

cc_library(
    name = "hello-time",
    srcs = ["hello-time.cc"],
    hdrs = ["hello-time.h"],
    # 让当前目标对于工作空间的main包可见。默认情况下目标仅仅被当前包可见
    visibility = ["//main:__pkg__"],
)

cc_library(
    name = "hello-greet",
    srcs = ["hello-greet.cc"],
    hdrs = ["hello-greet.h"],
)

cc_binary(
    name = "hello-world",
    srcs = ["hello-world.cc"],
    deps = [
        # 依赖当前包中的hello-greet目标
        ":hello-greet",
        # 依赖工作空间根目录下的lib包中的hello-time目标
        "//lib:hello-time",
    ],
)

在BUILD文件或者命令行中，你都使用标签（Label）来引用目标，其语法为：

//path/to/package:target-name

# 当引用当前包中的其它目标时，可以：
//:target-name
# 当引用当前BUILD文件中其它目标时，可以：
:target-name

workspace-name>/                          # 工作空间根目录
  bazel-my-project => <...my-project>     # execRoot的符号链接，所有构建动作在此目录下执行
  bazel-out => <...bin>                   # outputPath的符号链接
  bazel-bin => <...bin>                   # 最近一次写入的二进制目录的符号链接，即$(BINDIR)
  bazel-genfiles => <...genfiles>         # 最近一次写入的genfiles目录的符号链接，即$(GENDIR)



/home/user/.cache/bazel/                  # outputRoot，所有工作空间的Bazel输出的根目录
  _bazel_$USER/                           # outputUserRoot，当前用户的Bazel输出的根目录
    install/
      fba9a2c87ee9589d72889caf082f1029/   # installBase，Bazel安装清单的哈希值
        _embedded_binaries/               # 第一次运行时从Bazel可执行文件的数据段解开的可执行文件或脚本
    7ffd56a6e4cb724ea575aba15733d113/     # outputBase，某个工作空间根目录的哈希值
      action_cache/                       # Action cache目录层次
      action_outs/                        # Action output目录
      command.log                         # 最近一次Bazel命令的stdout/stderr输出
      external/                           # 远程存储库被下载、链接到此目录
      server/                             # Bazel服务器将所有服务器有关的文件存放在此
        jvm.out                           # Bazel服务器的调试输出
      execroot/                           # 所有Bazel Action的工作目录
        <workspace-name>/                 # Bazel构建的工作树
          _bin/                           # 助手工具链接或者拷贝到此
          bazel-out/                      # outputPath，构建的实际输出目录
            local_linux-fastbuild/        # 每个独特的BuildConfiguration实例对应一个子目录
              bin/                        # 单个构建配置二进制输出目录，$(BINDIR)
                foo/bar/_objs/baz/        # 命名为//foo/bar:baz的cc_*规则的Object文件所在目录
                  foo/bar/baz1.o          # //foo/bar:baz1.cc对应的Object文件
                  other_package/other.o   # //other_package:other.cc对应的Object文件
                foo/bar/baz               # //foo/bar:baz这一cc_binary生成的构件
                foo/bar/baz.runfiles/     # //foo/bar:baz生成的二进制构件的runfiles目录
                  MANIFEST
                  <workspace-name>/
                    ...
              genfiles/                   # 单个构建配置生成的源文件目录，$(GENDIR)
              testlogs/                   # Bazel的内部测试运行器将日志文件存放在此
              include/                    # 按需生成的include符号链接树，符号链接bazel-include指向这里
            host/                         # 本机的BuildConfiguration
        <packages>/                       # 构建引用的包，对于此包来说，它就像一个正常的WORKSPACE 

Bazel配置文件使用Starlark（原先叫Skylark）语言，具有短小、简单、线程安全的特点。

这种语言的语法和Python很类似，Starlark是Python2/Python3的子集。不支持的Python特性包括：

Starlark支持的数据类型包括：None、bool、dict、function、int、list、string，以及两种Bazel特有的类型：depset、struct。

# 定义一个数字
number = 18

# 定义一个字典
people = {
    "Alice": 22,
    "Bob": 40,
    "Charlie": 55,
    "Dave": 14,
}

names = ", ".join(people.keys())

# 定义一个函数
def greet(name):
  """Return a greeting."""
  return "Hello {}!".format(name)
# 调用函数
greeting = greet(names)


def fizz_buzz(n):
  """Print Fizz Buzz numbers from 1 to n."""
  # 循环结构
  for i in range(1, n + 1):
    s = ""
    # 分支结构
    if i % 3 == 0:
      s += "Fizz"
    if i % 5 == 0:
      s += "Buzz"
    print(s if s else i)

你可以在BUILD文件中声明和使用变量。使用变量可以减少重复的代码：

COPTS = ["-DVERSION=5"]

cc_library(
  name = "foo",
  copts = COPTS,
  srcs = ["foo.cc"],
)

cc_library(
  name = "bar",
  copts = COPTS,
  srcs = ["bar.cc"],
  deps = [":foo"],
)

如果要声明跨越多个BUILD文件共享的变量，必须把变量放入.bzl文件中，然后通过load加载bzl文件。

所谓Make变量，是一类特殊的、可展开的字符串变量，这种变量类似Shell中变量替换那样的展开。

Bazel提供了：

1. 预定义变量，可以在任何规则中使用
2. 自定义变量，在规则中定义。仅仅在依赖该规则的那些规则中，可以使用这些变量

仅仅那些标记为Subject to 'Make variable' substitution的规则属性，才可以使用Make变量。例如：

# 使用Make变量FOO
my_attr = "prefix $(FOO) suffix"
# 如果变量FOO的值为bar，则实际my_attr的值为prefix bar suffix

如果要使用$字符，需要用

$$

执行命令：

bazel info --show_make_env [build options]

任何规则可以使用以下变量：

下表中的变量可以在genrule规则的cmd属性中使用：

$@

$<

下表中的变量以Bazel的Label为参数，获取包的某类输入/输出路径：

cc_binary(
  name = "app",
  srcs = ["app.cc"]
)

bazel build //testapp:app

$(execpath :app)  # bazel-out/host/bin/testapp/app
$(execpath empty.source) # testapp/empty.source 

$(rootpath :app)  # testapp/app
$(rootpath empty.source)  # testapp/empty.source 

$(location :app) # bazel-out/host/bin/testapp/app
$(location empty.source) # testapp/empty.source 

为一组目标指定一个名字，你可以从其它规则中方便的引用这组目标。

Bazel鼓励使用filegroup，而不是直接引用目录。Bazel构建系统不能完全了解目录中文件的变化情况，因而文件发生变化时，可能不会进行重新构建。而使用filegroup，即使联用glob，目录中所有文件仍然能够被构建系统正确的监控。

示例：

filegroup(
    name = "exported_testdata",
    srcs = glob([
        "testdata/*.dat",
        "testdata/logs/**/*.log",
    ]),
)

要引用filegroup，只需要使用标签：

cc_library(
    name = "my_library",
    srcs = ["foo.cc"],
    data = [
        "//my_package:exported_testdata",
        "//my_package:mygroup",
    ],
)

定义一组测试用例，给出一个有意义的名称，便于在特定时机  —— 例如迁入代码、执行压力测试 —— 时执行这些测试用例。

示例：

# 匹配当前包中所有small测试
test_suite(
    name = "small_tests",
    tags = ["small"],
)
# 匹配不包含flaky标记的测试
test_suite(
    name = "non_flaky_test",
    tags = ["-flaky"],
)
# 指定测试列表
test_suite(
    name = "smoke_tests",
    tests = [
        "system_unittest",
        "public_api_unittest",
    ],
)

为规则设置一个别名：

filegroup(
    name = "data",
    srcs = ["data.txt"],
)
# 定义别名
alias(
    name = "other",
    actual = ":data",
)

通过匹配以Bazel标记或平台约束来表达的“配置状态”，config_setting能够触发可配置的属性。

下面这个例子，匹配针对ARM平台的构建：

config_setting(
    name = "arm_build",
    values = {"cpu": "arm"},
)

下面的例子，匹配任何定义了宏FOO=bar的针对X86平台的调试（-c dbg）构建：

config_setting(
    name = "x86_debug_build",
    values = {
        "cpu": "x86",
        "compilation_mode": "dbg",
        "define": "FOO=bar"
    },
)

下面的库，通过select来声明可配置属性：

cc_binary(
    name = "mybinary",
    srcs = ["main.cc"],
    deps = select({
        # 如果config_settings arm_build匹配正在进行的构建，则依赖arm_lib这个目标
        ":arm_build": [":arm_lib"],
        # 如果config_settings x86_debug_build匹配正在进行的构建，则依赖x86_devdbg_lib
        ":x86_debug_build": [":x86_devdbg_lib"],
        # 默认情况下，依赖generic_lib
        "//conditions:default": [":generic_lib"],
    }),
) 

一般性的规则 —— 使用用户指定的Bash命令，生成一个或多个文件。使用genrule理论上可以实现任何构建行为，例如压缩JavaScript代码。但是在执行C++、Java等构建任务时，最好使用相应的专用规则，更加简单。

不要使用genrule来运行测试，如果需要一般性的测试规则，可以考虑使用sh_test。

genrule在一个Bash shell环境下执行，当任意一个命令或管道失败（set -e -o pipefail），整个规则就失败。你不应该在genrule中访问网络。

示例：

genrule(
    name = "foo",
    # 不需要输入
    srcs = [],
    # 生成一个foo.h
    outs = ["foo.h"],
    # 运行当前规则所在包下的一个Perl脚本
    cmd = "./$(location create_foo.pl) > \"$@\"",
    tools = ["create_foo.pl"],
) 

隐含输出：

1. name.stripped，仅仅当显式要求才会构建此输出，针对生成的二进制文件运行strip -g以驱除debug符号。额外的strip选项可以通过命令行--stripopt=-foo传入
2. name.dwp，仅仅当显式要求才会构建此输出，如果启用了 Fission ，则此文件包含用于远程调试的调试信息，否则是空文件

属性列表：

--copt "-DENVOY_IGNORE_GLIBCXX_USE_CXX11_ABI_ERROR=1" 

linkopts=['-static']

导入预编译好的C/C++库。

属性列表：

对于所有cc_*规则来说，构建所需的任何头文件都要在hdrs或srcs中声明。

对于cc_library规则，在hdrs声明的头文件构成库的公共接口。这些头文件可以被当前库的hdrs/srcs中的文件直接包含，也可以被依赖（deps）当前库的其它cc_*的hdrs/srcs直接包含。位于srcs中的头文件，则仅仅能被当前库的hdrs/srcs包含。

cc_binary和cc_test不会暴露接口，因此它们没有hdrs属性。

属性列表：

可以使用Glob语法为目标添加多个文件：

cc_library(
    name = "build-all-the-files",
    srcs = glob(["*.cc"]),
    hdrs = glob(["*.h"]),
)

如果源码依赖于某个头文件，则该源码的规则需要dep头文件的库，仅仅直接依赖才需要声明：

# 三明治依赖面包
cc_library(
    name = "sandwich",
    srcs = ["sandwich.cc"],
    hdrs = ["sandwich.h"],
    # 声明当前包下的目标为依赖
    deps = [":bread"],
)
# 面包依赖于面粉，三明治间接依赖面粉，因此不需要声明
cc_library(
    name = "bread",
    srcs = ["bread.cc"],
    hdrs = ["bread.h"],
    deps = [":flour"],
)

cc_library(
    name = "flour",
    srcs = ["flour.cc"],
    hdrs = ["flour.h"],
)

有些时候你不愿或不能将头文件放到工作空间的include目录下，现有的库的include目录可能不符合

导入一个库，用于静态链接：

cc_import(
  name = "mylib",
  hdrs = ["mylib.h"],
  static_library = "libmylib.a",
  # 如果为1则libmylib.a总会链接到依赖它的二进制文件
  alwayslink = 1,
)

导入一个库，用于共享链接（UNIX）： 

cc_import(
  name = "mylib",
  hdrs = ["mylib.h"],
  shared_library = "libmylib.so",
)

通过接口库（Interface library）链接到共享库（Windows）：

cc_import(
  name = "mylib",
  hdrs = ["mylib.h"],
  # mylib.lib是mylib.dll的导入库，此导入库会传递给链接器
  interface_library = "mylib.lib",
  # mylib.dll在运行时需要，链接时不需要
  shared_library = "mylib.dll",
)

在二进制目标中选择链接到共享库还是静态库（UNIX）：

cc_import(
  name = "mylib",
  hdrs = ["mylib.h"],
  # 同时声明共享库和静态库
  static_library = "libmylib.a",
  shared_library = "libmylib.so",
)

# 此二进制目标链接到静态库
cc_binary(
  name = "first",
  srcs = ["first.cc"],
  deps = [":mylib"],
  linkstatic = 1, # default value
)

# 此二进制目标链接到共享库
cc_binary(
  name = "second",
  srcs = ["second.cc"],
  deps = [":mylib"],
  linkstatic = 0,
)

你可以在WORKSPACE中调用new_*存储库函数，来从网络中下载依赖。下面的例子下载Google Test库：

# 下载归档文件，并让其在工作空间的存储库中可用
new_http_archive(
    name = "gtest",
    url = "https://github.com/google/googletest/archive/release-1.7.0.zip",
    sha256 = "b58cb7547a28b2c718d1e38aee18a3659c9e3ff52440297e965f5edffe34b6d0",
    # 外部库的构建规则编写在gtest.BUILD
    # 如果此归档文件已经自带了BUILD文件，则可以调用不带new_前缀的函数
    build_file = "gtest.BUILD",
    # 去除路径前缀
    strip_prefix = "googletest-release-1.7.0",
)

构建此外部库的规则如下：

cc_library(
    name = "main",
    srcs = glob(
        # 前缀去除，原来是googletest-release-1.7.0/src/*.cc
        ["src/*.cc"],
        # 排除此文件
        exclude = ["src/gtest-all.cc"]
    ),
    hdrs = glob([
        # 前缀去除
        "include/**/*.h",
        "src/*.h"
    ]),
    copts = [
        # 前缀去除，原来是external/gtest/googletest-release-1.7.0/include
        "-Iexternal/gtest/include"
    ],
    # 链接到pthread
    linkopts = ["-pthread"],
    visibility = ["//visibility:public"],
)

沿用上面的例子，下面的目标使用gtest编写测试代码：

cc_test(
    name = "hello-test",
    srcs = ["hello-test.cc"],
    # 前缀去除
    copts = ["-Iexternal/gtest/include"],
    deps = [
        # 依赖gtest存储库的main目标
        "@gtest//:main",
        "//lib:hello-greet",
    ],
)

Bazel允许依赖其它项目中定义的目标，这些来自其它项目的依赖叫做“外部依赖“。当前工作空间的WORKSPACE文件声明从何处下载外部依赖的源码。

外部依赖可以有自己的1-N个BUILD文件，其中定义自己的目标。当前项目可以使用这些目标。例如下面的两个项目结构：

/
  home/
    user/
      project1/
        WORKSPACE
        BUILD
        srcs/
          ...
      project2/
        WORKSPACE
        BUILD
        my-libs/

如果project1需要依赖定义在project2/BUILD中的目标:foo，则可以在其WORKSPACE中声明一个存储库（repository），名字为project2，位于/home/user/project2。然后，可以在BUILD中通过标签@project2//:foo引用目标foo。

除了依赖来自文件系统其它部分的目标、下载自互联网的目标以外，用户还可以编写自己的存储库规则（repository rules ）以实现更复杂的行为。

WORKSPACE的语法格式和BUILD相同，但是允许使用不同的规则集。

Bazel会把外部依赖下载到

$(bazel info output_base)/external

bazel clean --expunge

可以使用local_repository、git_repository或者http_archive这几个规则来引用。

引用本地Bazel项目的例子：

local_repository(
    name = "coworkers_project",
    path = "/path/to/coworkers-project",
)

在BUILD中，引用coworkers_project中的目标//foo:bar时，使用标签@coworkers_project//foo:bar 

可以使用new_local_repository、new_git_repository或者new_http_archive这几个规则来引用。你需要自己编写BUILD文件来构建这些项目。

引用本地非Bazel项目的例子：

new_local_repository(
    name = "coworkers_project",
    path = "/path/to/coworkers-project",
    build_file = "coworker.BUILD",
)

cc_library(
    name = "some-lib",
    srcs = glob(["**"]),
    visibility = ["//visibility:public"],
) 

在BUILD文件中，使用标签@coworkers_project//:some-lib引用上面的库。 

对于Maven仓库，可以使用规则maven_jar/maven_server来下载JAR包，并将其作为Java依赖。

默认情况下，执行bazel Build时会按需自动拉取依赖，你也可以禁用此特性，并使用bazel fetch预先手工拉取依赖。

Bazel可以使用HTTPS_PROXY或HTTP_PROXY定义的代理地址。

Bazel会缓存外部依赖，当WORKSPACE改变时，会重新下载或更新这些依赖。

Bazel命令接收大量的参数，其中一部分很少变化，这些不变的配置项可以存放在.bazelrc中。

Bazel按以下顺序寻找.bazelrc文件：

1. 除非指定--nosystem_rc，否则寻找/etc/bazel.bazelrc
2. 除非指定--noworkspace_rc，否则寻找工作空间根目录的.bazelrc
3. 除非指定--nohome_rc，否则寻找当前用户的$HOME/.bazelrc

import %workspace%/tools/bazel.rc

build:memcheck --strip=never --test_timeout=3600

所谓Bazel扩展，是扩展名为.bzl的文件。你可以使用load语句加载扩展中定义的符号到BUILD中。

一次Bazel构建包含三个阶段：

1. 加载阶段：加载、eval本次构建需要的所有扩展、所有BUILD文件。宏在此阶段执行，规则被实例化。BUILD文件中调用的宏/函数，在此阶段执行函数体，其结果是宏里面实例化的规则被填充到BUILD文件中
2. 分析阶段：规则的代码——也就是它的implementation函数被执行，导致规则的Action被实例化，Action描述如何从输入产生输出
3. 执行阶段：执行Action，产生输出，测试也在此阶段执行

Bazel会并行的读取/解析/eval BUILD文件和.bzl文件。每个文件在每次构建最多被读取一次，eval的结果被缓存并重用。每个文件在它的全部依赖被解析之后才eval。加载一个.bzl文件没有副作用，仅仅是定义值和函数

宏（Macro）是一种函数，用来实例化（instantiates）规则。如果BUILD文件太过重复或复杂，可以考虑使用宏，以便减少代码。宏的函数在BUILD文件被读取时就立即执行。BUILD被读取（eval）之后，宏被替换为它生成的规则。bazel query只会列出生成的规则而非宏。

编写宏时需要注意：

1. 所有实例化规则的公共函数，都必须具有一个无默认值的name参数
2. 公共函数应当具有docstring
3. 在BUILD文件中，调用宏时name参数必须是关键字参数
4. 宏所生成的规则的name属性，必须以调用宏的name参数作为后缀
5. 大部分情况下，可选参数应该具有默认值None
6. 应当具有可选的visibility参数

要在宏中实例化原生规则（Native rules，不需要load即可使用的那些规则），可以使用native模块：

# 该宏实例化一个genrule规则
def file_generator(name, arg, visibility=None):
  // 生成一个genrule规则
  native.genrule(
    name = name,
    outs = [name + ".txt"],
    cmd = "$(location generator) %s > $@" % arg,
    tools = ["//test:generator"],
    visibility = visibility,
  )

使用上述宏的BUILD文件：

load("//path:generator.bzl", "file_generator")

file_generator(
    name = "file",
    arg = "some_arg",
)

执行下面的命令查看宏展开后的情况：

# bazel query --output=build //label

genrule(
  name = "file",
  tools = ["//test:generator"],
  outs = ["//test:file.txt"],
  cmd = "$(location generator) some_arg > $@",
)

规则（Rule）比宏更强大，能够对Bazel内部特性进行访问，并可以完全控制Bazel。

规则定义了为了产生输出，需要在输入上执行的一系列动作。例如，C++二进制文件规则以一系列.cpp文件为输入，针对输入调用g++，输出一个可执行文件。注意，从Bazel的角度来说，不但cpp文件是输入，g++、C++库也是输入。当编写自定义规则时，你需要注意，将执行Action所需的库、工具作为输入看待。

Bazel内置了一些规则，这些规则叫原生规则，例如cc_library、cc_library，对一些语言提供了基础的支持。通过编写自定义规则，你可以实现对任何语言的支持。

定义在.bzl中的规则，用起来就像原生规则一样 —— 规则的目标具有标签、可以出现在bazel query。

规则在分析阶段的行为，由它的implementation函数决定。此函数不得调用任何外部工具，它只是注册在执行阶段需要的Action。

在.bzl文件中，你可以调用rule创建自定义规则，并将其保存到全局变量：

def _empty_impl(ctx):
    # 分析阶段此函数被执行
    print("This rule does nothing")

empty = rule(implementation = _empty_impl)

然后，规则可以通过load加载到BUILD文件：

load("//empty:empty.bzl", "empty")

# 实例化规则
empty(name = "nothing")

属性即实例化规则时需要提供的参数，例如srcs、deps。在自定义规则的时候，你可以列出所有属性的名字和Schema：

sum = rule(
    implementation = _impl,
    attrs = {
        # 定义一个整数属性，一个列表属性
        "number": attr.int(default = 1),
        "deps": attr.label_list(),
    },
)

实例化规则的时候，你需要以参数的形式指定属性：

sum(
    name = "my-target",
    deps = [":other-target"],
)

sum(
    name = "other-target",
)

如果实例化规则的时候，没有指定某个属性的值（且没指定默认值），规则的实现函数会在ctx.attr中看到一个占位符，此占位符的值取决于属性的类型。

使用default为属性指定默认值，使用 mandatory=True 声明属性必须提供。

任何规则自动具有以下属性：deprecation, features, name, tags, testonly, visibility。

任何测试规则具有以下额外属性：args, flaky, local, shard_count, size, timeout。

有两类特殊属性需要注意：

1. 依赖属性：例如attr.label、attr.label_list，用于声明拥有此属性的目标所依赖的其它目标
2. 输出属性：例如attr.output、attr.output_list，声明目标的输出文件，较少使用

上面两类属性的值都是Label类型。 

具有默认值的依赖属性，称为隐含依赖（implicit dependency）。如果要硬编码规则和工具（例如编译器）之间的关系，可通过隐含依赖。从规则的角度来看，这些工具仍然属于输入，就像源代码一样。

某些情况下，我们会为规则添加具有默认值的属性，同时还想禁止用户修改属性值，这种情况下可以使用私有属性。

私有属性以下划线 _ 开头，必须具有默认值。

实例化规则不会返回值，但是会定义一个新的目标。

任何规则都需要提供一个实现函数。提供在分析阶段需要严格执行的逻辑。此函数不能有任何读写行为，仅仅用于注册Action。

实现函数具有唯一性入参  —— 规则上下文，通常将其命名为ctx。通过规则上下文你可以：

1. 访问规则属性
2. 获得输入输出文件的handle
3. 创建Actions
4. 通过providers向依赖于当前规则的其它规则传递信息

规则上下文对象的具有以下主要方法或属性：

def _impl(ctx):
    # The list of arguments we pass to the script.
    args = [ctx.outputs.out.path] + [f.path for f in ctx.files.chunks]

    # 调用可执行文件
    ctx.actions.run(
        inputs = ctx.files.chunks,
        outputs = [ctx.outputs.out],
        arguments = args,
        progress_message = "Merging into %s" % ctx.outputs.out.short_path,
        executable = ctx.executable._merge_tool,
    )
// 规则定义
concat = rule(
    implementation = _impl,
    attrs = {
        "chunks": attr.label_list(allow_files = True),
        "out": attr.output(mandatory = True),
        "_merge_tool": attr.label(
            executable = True,
            cfg = "host",
            allow_files = True,
            default = Label("//actions_run:merge"),
        ),
    },
) 

def _impl(ctx):
    output = ctx.outputs.out
    input = ctx.file.file

    # 访问inputs中声明的文件
    ctx.actions.run_shell(
        inputs = [input],
        outputs = [output],
        progress_message = "Getting size of %s" % input.short_path,
        command = "stat -L -c%%s '%s' > '%s'" % (input.path, output.path),
    )

# 规则定义
size = rule(
    implementation = _impl,
    attrs = {"file": attr.label(mandatory = True, allow_single_file = True)},
    outputs = {"out": "%{name}.size"},
)

attr.label(executable=True)

string ctx.expand_location(input, targets=[])

attr.labe(allow_single_file=True)

list(ctx.attr.<ATTR>.files)[0]

存储库规则用于定义外部存储库。外部存储库是一种规则，这种规则只能用在WORKSPACE文件中，可以在Bazel加载阶段启用非封闭性（ non-hermetic，所谓封闭是指自包含，不依赖于外部环境）操作。每个外部存储库都创建自己的WORKSPACE，具有自己的BUILD文件和构件。

外部存储库可以用来：

1. 加载第三方依赖，例如Maven打包的库
2. 为运行构件的主机生成特化的BUILD文件

在bzl文件中，调用repository_rule函数可以创建一个存储库规则，你需要将其存放在全局变量中：

local_repository = repository_rule(
    # 实现函数
    implementation=_impl,
    local=True,
    # 属性列表
    attrs={"path": attr.string(mandatory=True)}) 

每个存储库规则都必须提供实现函数，其中包含在Bazel加载阶段需要执行的严格的逻辑。该函数具有唯一的入参repository_ctx：

def _impl(repository_ctx):
  # 你可以通过repository_ctx访问属性值、调用非密封性函数（例如查找、执行二进制文件，创建或下载文件到存储库）
  repository_ctx.symlink(repository_ctx.attr.path, "") 

引用存储库中规则时，可以使用

@REPO_NAMAE//package:target

存储库规则上下文对象的具有以下主要方法或属性：

struct repository_ctx.download(url, output='', sha256='', executable=False)

# from 符号链接的源，string/Label/path类型
# to 相对于存储库目录的符号链接文件的路径
None repository_ctx.symlink(from, to)

# 构建foo/bar包中的wiz目标
bazel build //foo/bar:wiz
# 构建foo/bar包中的bar目标
bazel build //foo/bar
# 构建foo/bar包中的所有规则
bazel build //foo/bar:all
# 构建foo目录下所有子代包的所有规则
bazel build //foo/...
bazel build //foo/...:all
# 构建foo目录下所有子代包的所有目标（规则和文件）
bazel build //foo/...:*
bazel build //foo/...:all-targets

//

tags = ["manual"]

--fetch=false

# 抓取两个外部依赖
bazel fetch //foo:bar //bar:baz
# 抓取工作空间的全部外部依赖
bazel fetch //...

The post Bazel学习笔记 appeared first on 绿色记忆.