
## What is Flyway

[Flyway](https://flywaydb.org/) is an open-source database version control tool implemented in Java. It offers various methods such as CLI, Java API, Maven/Gradle Plugin, etc., making it convenient for developers to manage and deploy changes to database table structures and contents in traceable code form. Flyway supports most mainstream relational databases, including MySQL, SQL Server, Oracle Database, PostgreSQL, SQLite, TiDB, MariaDB, etc. Support for MongoDB is still in the preview stage (Mongock is recommended instead). Its counterpart, Liquibase, is more complex to use compared to Flyway, with additional concepts serving as a learning curve.

<div>
  <github user="flyway" repo="flyway"></github>
</div>

## Integration with Spring Boot

Following Spring Boot's well-known convention-over-configuration principle, the Spring team provides [Flyway Auto Configuration](https://github.com/spring-projects/spring-boot/tree/main/spring-boot-project/spring-boot-autoconfigure/src/main/java/org/springframework/boot/autoconfigure/flyway).

```java {2-4} showLineNumbers
@AutoConfiguration(after = { DataSourceAutoConfiguration.class, JdbcTemplateAutoConfiguration.class,HibernateJpaAutoConfiguration.class })
@ConditionalOnClass(Flyway.class)
@Conditional(FlywayDataSourceCondition.class)
@ConditionalOnProperty(prefix = "spring.flyway", name = "enabled", matchIfMissing = true)
@Import(DatabaseInitializationDependencyConfigurer.class)
@ImportRuntimeHints(FlywayAutoConfigurationRuntimeHints.class)
public class FlywayAutoConfiguration { /*...*/ }
```

The fully qualified class name `org.springframework.boot.autoconfigure.flyway.FlywayAutoConfiguration` will perform automatic configuration when the following conditions are detected:

1. The `Flyway` class exists (class name: `org.flywaydb.core.Flyway`)
2. Bean or properties conditions are met in the `FlywayDataSourceCondition` class, meaning connection configurations such as `DataSource` and database URL exist.
3. The property configuration `spring.flyway.enabled=true` exists.

For MySQL, besides the required JDBC dependency, you need to include Flyway's own dependencies. Spring Boot has already declared the version number in the POM, so no additional version field is needed here:

```xml
<dependency>
    <groupId>org.flywaydb</groupId>
    <artifactId>flyway-core</artifactId>
</dependency>
<dependency>
    <groupId>org.flywaydb</groupId>
    <artifactId>flyway-mysql</artifactId>
</dependency>
```

There are two ways to use it:

1. Native SQL
2. Java API

Flyway will discover version change files or implementation classes based on the default configured folder path `classpath:db/migration`.

```shell
V{VERSION}__{DESCRIPTION}.sql
V{VERSION}__{DESCRIPTION}.java
```

`VERSION` can be a string version containing dots and underscores, and `DESCRIPTION` is a simple description text. The separator in the middle is two underscores. Custom prefixes and suffixes can be configured through spring.flyway.

The `V{VERSION}__{DESCRIPTION}.sql` file needs to be placed in the `src/main/resources/db/migration` folder, while Java classes need to define the package name as db.migration and inherit the superclass `BaseJavaMigration`. We need to do is overriding the following method:

```java
void migrate(Context context) throws Exception;
```

According to the official documentation, we can implement database content version changes using Java code in the `migrate()` method. However, the drawback here is that we cannot perform dependency injection for `V{VERSION}__{DESCRIPTION}.java` in the Spring environment. We can only attempt to use plain SQL implementations through native JDBC methods, static methods, etc., rather than fully utilizing existing DAO interfaces for business data updates. Therefore, custom configuration of Flyway is required.

## Performing Dependency Injection into Flyway Class

```java
package me.lawrenceli.migration.config;

import jakarta.annotation.PostConstruct;
import org.flywaydb.core.Flyway;
import org.flywaydb.core.api.migration.JavaMigration;
import org.flywaydb.core.api.output.MigrateResult;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.Configuration;

import javax.sql.DataSource;

@Configuration
public class FlywayConfig {
    // Custom migration history table name
    private static final String SCHEMA_HISTORY_TABLE = "schema_changes";

    @Autowired
    private DataSource dataSource;

    @Autowired
    private ApplicationContext applicationContext;

    @PostConstruct
    public void migrate() {
        log.info("Flyway, starting!");
        // Retrieving all migration implementation classes through the Spring container
        // All implementation classes no longer need to be defined in the `db.migration` package,
        // they can be placed in any location that supports Bean scanning
        JavaMigration[] migrationBeans = applicationContext
                .getBeansOfType(JavaMigration.class)
                .values()
                .toArray(new JavaMigration[0]);
        Flyway flyway = Flyway.configure()
                .dataSource(dataSource) // No need to configure flyway JDBC URL by using the original DataSource
                .locations("db/migration") // Default migration script path
                .table(SCHEMA_HISTORY_TABLE) // Default migration history table is `flyway_schema_history`
                .baselineOnMigrate(true) // Default false, must be set to true for initial migration on existing databases
                .baselineVersion("0") // Default "1"
                .executeInTransaction(true) // Performing migration as a transaction
                .installedBy(applicationContext.getId()) // Using microservice name as migration executor
                .javaMigrations(migrationBeans) // Registering migration classes
                .load();
        MigrateResult migrate = flyway.migrate(); // Executing migration by calling subclass implementations sequentially
        log.info("Flyway migrated {} versions. {}", migrate.migrationsExecuted, migrate.success);
    }
}

```

Since Flyway's configuration is based on manual configuration like this, it's necessary to exclude the original auto-configuration classes in the Spring Boot startup class to avoid conflicts in auto-configuration loading.

```java
@SpringBootApplication(exclude = FlywayAutoConfiguration.class)
public class MyApplication {

    public static void main(String[] args) {
        SpringApplication.run(MyApplication.class, args);
    }

}
```

Finally, define a Component Bean to implement `BaseJavaMigration`:

```java {12} showLineNumbers caption="Example Migration Component"
package me.lawrenceli.balabala.migration;

import org.flywaydb.core.api.migration.BaseJavaMigration;
import org.flywaydb.core.api.migration.Context;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

@Component
public class V2__QueryExample extends BaseJavaMigration {

    @Autowired
    private MyMapper myMapper; // Bean of DAO

    @Override
    public void migrate(Context context) throws Exception {
        Data data = myMapper.selectById(2024L);
        // ... other CRUD codes with Java
    }
}
```

Now, all migration classes can easily use dependency injection to perform CRUD operations. In practice, Flyway will execute all migrations synchronously after database connection configuration and before HTTP service exposure, ensuring that timing does not affect online services.

## Reference

- <https://github.com/flyway/flyway/issues/1062>


Increase reliability of deployments by versioning your database

Dependency Injection with Flyway


## Flyway 是什么

[Flyway](https://flywaydb.org/) 是一款开源的，基于 Java 实现的数据库内容变更控制工具。它提供了 CLI、Java API、Maven/Gradle Plugin 等多种方式方便开发人员将数据库的表结构改动、内容变动以可追溯的代码形式进行管理和部署。Flyway 支持包括大多数主流的关系型数据库：MySQL、SQL Server、Oracle Database、PostgreSQL、SQLite、TiDB、MariaDB 等，对于 MongoDB 的支持尚在预览阶段（更推荐 Mongock），同类竞品有 Liquibase，但 Liquibase 的使用相比 Flyway 更复杂，有额外的概念作为学习成本。

<div>
  <github user="flyway" repo="flyway"></github>
</div>

## 配合 Spring Boot 使用

遵循 SpringBoot 广为人知的约定大于配置，Spring 官方提供了 [Flyway 的自动配置实现](https://github.com/spring-projects/spring-boot/tree/main/spring-boot-project/spring-boot-autoconfigure/src/main/java/org/springframework/boot/autoconfigure/flyway)。

```java {2-4} showLineNumbers
@AutoConfiguration(after = { DataSourceAutoConfiguration.class,JdbcTemplateAutoConfiguration.class,HibernateJpaAutoConfiguration.class })
@ConditionalOnClass(Flyway.class)
@Conditional(FlywayDataSourceCondition.class)
@ConditionalOnProperty(prefix = "spring.flyway", name = "enabled", matchIfMissing = true)
@Import(DatabaseInitializationDependencyConfigurer.class)
@ImportRuntimeHints(FlywayAutoConfigurationRuntimeHints.class)
public class FlywayAutoConfiguration { /*...*/ }
```

全限定类名 `org.springframework.boot.autoconfigure.flyway.FlywayAutoConfiguration` 会在检测到以下几种状态下实现自动配置：

1. 存在 `Flyway` 类 (即 `org.flywaydb.core.Flyway`)
2. 满足 `FlywayDataSourceCondition` 类中的 Bean 或 Properties 条件，其实就是 `DataSource` 和数据库 URL 等连接配置存在
3. 存在 `spring.flyway.enabled=true` 属性配置

对于 MySQL 而言，除了必须的 JDBC 依赖之外，需要引入 Flyway 自身依赖，SpringBoot 已经在 pom 中声明过版本号，因此此处无需额外定义版本字段:

```xml
<dependency>
    <groupId>org.flywaydb</groupId>
    <artifactId>flyway-core</artifactId>
</dependency>
<dependency>
    <groupId>org.flywaydb</groupId>
    <artifactId>flyway-mysql</artifactId>
</dependency>
```

具体使用有两种方式：

1. 原生 SQL
2. Java API

Flyway 会基于默认配置的文件夹路径 classpath:db/migration 发现版本变更文件或实现类。

```shell
V{VERSION}__{DESCRIPTION}.sql
V{VERSION}__{DESCRIPTION}.java
```

`VERSION` 可使用包含小数点、下划线的字符串版本，`DESCRIPTION` 则是简单的描述文本。中间的分割符是两个下划线。可通过 spring.flyway 配置自定义前后缀。

`V{VERSION}__{DESCRIPTION}.sql` 文件需要存放在 src/main/resources/db/migration 文件夹下; Java 类则需要定义包名 `db.migration` 并且继承父类 `BaseJavaMigration`，重写如下方法:

```java
void migrate(Context context) throws Exception;
```

按照官方的写法，我们就可以在 `migrate()` 方法中去用 Java 代码实现数据库内容版本变动了，但是这里有一个缺点，那就是我们在 Spring 环境下无法针对 `V{VERSION}__{DESCRIPTION}.java` 进行依赖注入，只能尝试通过原生 JDBC 的方式、静态方法等去写比较朴素的 SQL 实现，而不能充分成分利用现有的 DAO 接口来做业务数据的更新。因此需要自定义配置 Flyway。

## 向 Flyway Migration 实现类进行依赖注入

```java
package me.lawrenceli.migration.config;

import jakarta.annotation.PostConstruct;
import org.flywaydb.core.Flyway;
import org.flywaydb.core.api.migration.JavaMigration;
import org.flywaydb.core.api.output.MigrateResult;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.Configuration;

import javax.sql.DataSource;

@Configuration
public class FlywayConfig {
    // 自定义迁移历史表名
    private static final String SCHEMA_HISTORY_TABLE = "schema_changes";

    @Autowired
    private DataSource dataSource;

    @Autowired
    private ApplicationContext applicationContext;

    @PostConstruct
    public void migrate() {
        log.info("Flyway, 启动!");
        // 通过 Spring 容器获取所有迁移实现类
        // 这样一来，所有实现类就不再需要定义在 package `db.migration` 下，可以放在任何支持 Bean 扫描的位置。
        JavaMigration[] migrationBeans = applicationContext
                .getBeansOfType(JavaMigration.class)
                .values()
                .toArray(new JavaMigration[0]);
        Flyway flyway = Flyway.configure()
                .dataSource(dataSource) // 通过原本的 DataSource Bean 实现无需配置 flyway 自身的 JDBC URL
                .locations("db/migration") // 默认迁移脚本路径
                .table(SCHEMA_HISTORY_TABLE) // 默认迁移历史表为 `flyway_schema_history`
                .baselineOnMigrate(true) // 默认 false, 对以存在的数据库做首次迁移必须设置开启
                .baselineVersion("0") // 默认 "1"
                .executeInTransaction(true) // 将迁移作为事务，你懂的
                .installedBy(applicationContext.getId()) // 将微服务名作为迁移执行者
                .javaMigrations(migrationBeans) // 注册迁移类
                .load();
        MigrateResult migrate = flyway.migrate(); // 执行迁移，依次调用子类实现
        log.info("Flyway 迁移了 {} 版. {}", migrate.migrationsExecuted, migrate.success);
    }
}
```

由于 Flyway 的配置基于这种手动配置，因此需要在 SpringBoot 启动类上排除原有的自动配置类，以防止自动配置存在加载冲突。

```java
@SpringBootApplication(exclude = FlywayAutoConfiguration.class)
public class MyApplication {

    public static void main(String[] args) {
        SpringApplication.run(MyApplication.class, args);
    }

}
```

最后，定义一个 Component Bean 去实现 `BaseJavaMigration`:

```java {12} showLineNumbers caption="Example Migration Component"
package me.lawrenceli.balabala.migration;

import org.flywaydb.core.api.migration.BaseJavaMigration;
import org.flywaydb.core.api.migration.Context;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

@Component
public class V2__QueryExample extends BaseJavaMigration {

    @Autowired
    private MyMapper myMapper; // Bean of DAO

    @Override
    public void migrate(Context context) throws Exception {
        Data data = myMapper.selectById(2024L);
        // ... other CRUD codes with Java
    }
}
```

这样，所有的迁移类都可以方便地使用依赖注入来愉快地做 CRUD 了。经过实践，Flyway 会在数据库连接配置后、HTTP 服务暴露(也就是 Servlet 容器监听端口)前同步地执行完所有迁移，因此无需担心执行时机影响线上服务。

## 参考

- <https://github.com/flyway/flyway/issues/1062>


在 Flyway 迁移类中实现依赖注入


基于 Spring Cloud 使用一致性哈希算法实现分布式 WebSocket. / 基于 RabbitMQ 广播实现分布式 WebSocket.

### WebSocket 协议

参考 [RFC](https://tools.ietf.org/html/rfc6455) 标准规范。

### 思路

核心问题在于具备 WebSocket 服务端的 SpringBoot 或 SpringCloud 等微服务应用是**有状态**的。服务端的内存中存在维持连接的
Session。客户端连接服务器时，只能和集群中的唯一一个服务实例来连接，而且数据传输过程中也直接和此实例通信。

解决方案有很多。

#### 方案一：将状态放到应用之外的存储中，但不可行

这个方案非常类似于 HTTP 协议的 Token 实现（非 JWT），Java 中具体的例子就是 Spring Session 以及 Spring Security 的 Token Store，比如 RedisTokenStore 实现了将
OAuth2 Token 存放到 Redis 的逻辑。此方案的核心前提是可序列化。WebSocket 协议和 HTTP 协议并不同，一个 WebSocket Session 对应于一次 TCP Socket
连接而非数据，新的连接将会基于新的四元组，不可能做到序列化。

#### 方案二：一致性哈希

像微服务注册中心一样，维护一个全局映射关系 —— 当前客户端的 WebSocket
连接是和哪一个实例/节点保持的。每次通信都经过此注册中心或哈希，查询到具体的服务实例，将连接交给它。同样会遇到一些问题，比如：哈希环的实现、如何解决一致性问题（单节点故障的 Session 迁移重连将代价降到最低）、CAP
取舍、实例的区分依据是什么（IP ? 实例
ID？）、可能与注册中心构成强依赖、以及部署环境的变更带来运维上的挑战等难题。这个方案实施有一定的成本和技术限制，可以参考[这篇文章](https://segmentfault.com/a/1190000017307713)的大体思路，理解了核心的哈希环的概念(如下图)，再配合注册中心、服务上下线消息，就能完成基本的实现。我利用
Nacos、Redis、RabbitMQ 基于 Spring Cloud 写了一个通过一致性哈希来实现的全栈项目，前端和后端都已开源 ——
[WebSocket 集群的一致性哈希实践(Spring Cloud 后端) - GitHub](https://github.com/la3rence/websocket-cluster)
, [WebSocket 客户端模拟与服务列表(React 前端) - GitHub](https://github.com/la3rence/websocket-cluster-front),
最近刚刚写好基本的功能，参考 [@ufiredong](https://github.com/ufiredong)
的方式利用 [Java 语言的 Docker SDK](https://github.com/docker-java/docker-java) 实现模拟 WebSocket 实例的上下线。未来还会稍微做细微的优化。

<div>
    <github user="la3rence" repo="websocket-cluster"></github>
</div>

![HashRing: 虚拟节点上线后部分客户端需要迁移节点](/images/websocket-cluster/hashring.png)

另外，Nacos 作为微服务的注册中心是国内比较流行的选择。但有人早就向 Nacos 社区提出了[维护哈希环的议题](https://github.com/alibaba/nacos/issues/2114)；姑且不说此议题是否合理，但
Nacos 的维护人员竟然没有能够理解这样的需求，还是有点令人诧异。

最终效果如下, 上线一个新的服务实例后，客户端会自动根据最新哈希环重新路由到最新节点上，收发消息一切正常：

<div>
  <bilibili bv="BV1sg411q78V" />
</div>

#### 方案三：广播队列

每次连接都需要通知到所有实例，实例各自都判断连接状态在不在自己这里，不在的话直接忽略，在就处理。类似于发布订阅模型，可以通过 MQ（RabbitMQ、Kafka、RocketMQ 等）或 Redis
做到。此方案实现简单，适用于集群规模不大的场景，因为需要所有的节点进行判断或计算。我愿称之为：分布式事件驱动。

以下基于 RabbitMQ 做了简单实现。

##### 简单广播实现 WebSocket 集群

[声明式 API](/blog/declarative-programming) 的好处就体现在这里：短短几行就直接利用了 RabbitMQ 的 Java SDK 创建好交换机和队列以及绑定关系。

```java
package me.lawrenceli.websocket.server.configuration;

import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.core.AnonymousQueue;
import org.springframework.amqp.core.Binding;
import org.springframework.amqp.core.BindingBuilder;
import org.springframework.amqp.core.FanoutExchange;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Slf4j
@Configuration
public class MqConfig {

    private static final String FANOUT_EXCHANGE = "websocket-exchange";

    @Bean
    public FanoutExchange fanoutExchange() {
        log.info("创建广播交换机 [{}]", FANOUT_EXCHANGE);
        return new FanoutExchange(FANOUT_EXCHANGE);
    }

    @Bean
    public AnonymousQueue queueForWebSocket() {
        log.info("创建用于 WebSocket 的匿名队列");
        return new AnonymousQueue();
    }

    /**
     * @param fanoutExchange    交换机
     * @param queueForWebSocket 队列
     * @return Binding
     */
    @Bean
    public Binding bindingSingle(FanoutExchange fanoutExchange, AnonymousQueue queueForWebSocket) {
        log.info("把匿名队列 [{}] 绑定到广播交换器 [{}]", queueForWebSocket.getName(), fanoutExchange.getName());
        return BindingBuilder.bind(queueForWebSocket).to(fanoutExchange);
    }

}
```

然后是生产者和消费者。生产者将消息发送到队列中进行广播，集群的消费者监听队列，判断此 WebSocket Session 是否在当前消费节点再做进一步处理。

消息生产方，一般是外层接收到消息通信请求，然后调用进行集群内广播：

```java
package me.lawrenceli.websocket.server.configuration;

import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.core.FanoutExchange;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

@Slf4j
@Component
public class FanoutSender {

    @Autowired
    private FanoutExchange fanoutExchange;

    @Autowired
    private RabbitTemplate rabbitTemplate;

    public void send(Object message) {
        log.info("开始发送广播: [{}]", message.toString());
        rabbitTemplate.convertAndSend(fanoutExchange.getName(), "", message);
    }

}
```

消息消费方，多节点共同消费同一消息，区分是否需要自己来处理：

```java
package me.lawrenceli.websocket.server.configuration;

import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;

@Slf4j
@Component
public class FanoutReceiver {

    @RabbitListener(queues = "#{queueForWebSocket.name}")
    public void singleReceiver(Object message) {
        log.info("队列接收到了消息: [{}]", message.toString());
        // 判断 WebSocket Session 是否在当前节点
        // 一般维护在一个 ConcurrentHashMap 静态变量中，这里叫 sessionMap
        // message 中有类似 SessionId 的字段
        if (sessionMap.contains(sessionId)) {
            log.info("WebSocket Session 在当前节点");
            // 执行相应的流程
        } else {
            log.info("当前节点无此 WebSocket Session");
            // 什么都不用做，直接忽略
        }
    }

}
```

注意，方案 2、方案 3 都没有解决单节点故障问题，因为从本质上讲，没有实现 WebSocket Session
的全局共享。但是方案二能够依赖哈希环的更新，最大程度上地降低迁移连接的代价，客户端会有重试的机制来针对单节点故障问题做容错。如果规模较大，项目复杂，且客户端连接数多，推荐使用一致性哈希的方案二，当然下面也有其他的社区方案。

### 其他案例

即刻 App 后端基于 Node.js 使用了 [socket.io](http://socket.io) 实现多端实时通信，Joway
的[这篇文章](https://blog.joway.io/posts/socket-io/)交代的一些细节也涉及了分布式部署的问题。

socket.io 官网也提供了[多节点的使用方法](https://socket.io/docs/v3/using-multiple-nodes/index.html)，不外乎 IP Hash 之类的方案。

### Kubernetes 内的 Websocket 集群

如果部署环境基于 Kubernetes 内的 WebSocket
集群，也可以参考[阳明的这篇文章](https://www.qikqiak.com/post/socketio-multiple-nodes-in-kubernetes/)来实践 （利用 Service
的 `sessionAffinity` 会话亲和确保每次将来自特定客户端的连接传递到同一 Pod：使用 `ClientIP` 声明）。


WebSocket 在 Spring Cloud 中如何实践分布式集群：一致性哈希的实现和 Fanout 广播

B!og

Tagged with `java`

Tagged with java

Tagged with `java`