Debezium MySQL
Một số lỗi có thể gặp
- Lỗi Unable to connect: Plugin 'mysql_native_password' is not loaded

Debezium MySQL

Binlog (binary log)

Kiểm tra xem MySQL đã bật chế độ ghi log nhị phân hay chưa bằng lệnh:

SHOW VARIABLES LIKE 'log_bin';

Nếu kết quả là 'ON', binlog đang được bật.

Liệt kê các tệp binlog:

SHOW BINARY LOGS;

Xem nội dung binlog:

tungnt@MacBook-Pro-cua-Nguyen-2 mysql % mysqlbinlog /opt/homebrew/var/mysql/binlog.000217
# The proper term IS pseudo_replica_mode, but we USE this compatibility alias
# TO make the statement usable ON server versions 8.0.24 AND older.
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#241021 13:19:55 server id 1  end_log_pos 127 CRC32 0x609227a3 	START: binlog v 4, server v 8.4.2 created 241021 13:19:55 at startup
# Warning: this binlog IS either IN USE OR was NOT closed properly.
ROLLBACK/*!*/;
BINLOG '
i/IVZw8BAAAAewAAAH8AAAABAAQAOC40LjIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAACL8hVnEwANAAgAAAAABAAEAAAAYwAEGggAAAAAAAACAAAACgoKKioAEjQA
CigAAAGjJ5Jg
'/*!*/;
# at 127
#241021 13:19:55 server id 1  end_log_pos 158 CRC32 0xcf101e2a 	Previous-GTIDs
# [empty]
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# END OF log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;

Cài đặt Kafka và Zookeeper

https://kafka.apache.org/downloads

Khởi động Zookeeper:

% zookeeper-server-start config/zookeeper.properties

Khởi động Kafka:

% kafka-server-start config/server.properties

Cài đặt Debezium

https://debezium.io/releases/

wget https://repo1.maven.org/maven2/io/debezium/debezium-connector-mysql/1.9.5.Final/debezium-connector-mysql-1.9.5.Final-plugin.tar.gz
 
tar -xvf debezium-connector-mysql-1.9.5.Final-plugin.tar.gz
 
mkdir kafka_2.12-3.8.0/plugins/debezium-connector-mysql
 
cp -r debezium-connector-mysql kafka_2.12-3.8.0/plugins/debezium-connector-mysql

Cấu hình Kafka Connect

vim kafka_2.12-3.8.0/config/connect-standalone.properties
 
bootstrap.servers=localhost:9092
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
offset.storage.file.filename=/tmp/connect.offsets
plugin.path=/Users/tungnt/Downloads/Libs/kafka_2.12-3.8.0/plugins

vim kafka_2.12-3.8.0/config/mysql-connector.properties
 
name=mysql-connector
connector.class=io.debezium.connector.mysql.MySqlConnector
database.hostname=localhost
database.port=3306
database.user=xxx
database.password=xxx
database.server.id=184054
database.server.name=dbserver1
database.include.list=testdb
database.history.kafka.bootstrap.servers=localhost:9092
database.history.kafka.topic=schema-changes.testdb

Khởi động Kafka Connect và Debezium

./bin/connect-standalone.sh config/connect-standalone.properties config/mysql-connector.properties

Một số lỗi có thể gặp

Lỗi Unable to connect: Plugin 'mysql_native_password' is not loaded

Lỗi Unable to connect: Plugin 'mysql_native_password' is not loaded xảy ra khi MySQL đang sử dụng một cơ chế xác thực khác (ví dụ: caching_sha2_password) thay vì mysql_native_password. Điều này thường gặp phải khi bạn sử dụng MySQL 8.0 trở lên, trong đó mặc định là caching_sha2_password, nhưng Debezium yêu cầu mysql_native_password để kết nối.

mysql -u root -p

Kiểm tra xem plugin mysql_native_password đã được cài đặt hay chưa:

mysql> SHOW PLUGINS;
+----------------------------------+----------+--------------------+---------+---------+
| Name                             | STATUS   | TYPE               | Library | License |
+----------------------------------+----------+--------------------+---------+---------+
| binlog                           | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| sha256_password                  | ACTIVE   | AUTHENTICATION     | NULL    | GPL     |
| caching_sha2_password            | ACTIVE   | AUTHENTICATION     | NULL    | GPL     |
| sha2_cache_cleaner               | ACTIVE   | AUDIT              | NULL    | GPL     |
| daemon_keyring_proxy_plugin      | ACTIVE   | DAEMON             | NULL    | GPL     |
| CSV                              | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| MEMORY                           | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| InnoDB                           | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| INNODB_TRX                       | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_CMP                       | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_CMP_RESET                 | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_CMPMEM                    | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_CMPMEM_RESET              | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_CMP_PER_INDEX             | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_CMP_PER_INDEX_RESET       | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_BUFFER_PAGE               | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_BUFFER_PAGE_LRU           | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_BUFFER_POOL_STATS         | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_TEMP_TABLE_INFO           | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_METRICS                   | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_FT_DEFAULT_STOPWORD       | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_FT_DELETED                | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_FT_BEING_DELETED          | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_FT_CONFIG                 | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_FT_INDEX_CACHE            | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_FT_INDEX_TABLE            | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_TABLES                    | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_TABLESTATS                | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_INDEXES                   | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_TABLESPACES               | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_COLUMNS                   | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_VIRTUAL                   | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_CACHED_INDEXES            | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_SESSION_TEMP_TABLESPACES  | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| MyISAM                           | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| MRG_MYISAM                       | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| PERFORMANCE_SCHEMA               | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| TempTable                        | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| ARCHIVE                          | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| BLACKHOLE                        | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| FEDERATED                        | DISABLED | STORAGE ENGINE     | NULL    | GPL     |
| ndbcluster                       | DISABLED | STORAGE ENGINE     | NULL    | GPL     |
| ndbinfo                          | DISABLED | STORAGE ENGINE     | NULL    | GPL     |
| ndb_transid_mysql_connection_map | DISABLED | INFORMATION SCHEMA | NULL    | GPL     |
| ngram                            | ACTIVE   | FTPARSER           | NULL    | GPL     |
| mysqlx_cache_cleaner             | ACTIVE   | AUDIT              | NULL    | GPL     |
| mysqlx                           | ACTIVE   | DAEMON             | NULL    | GPL     |
| mysql_native_password            | DISABLED | AUTHENTICATION     | NULL    | GPL     |
+----------------------------------+----------+--------------------+---------+---------+
48 ROWS IN SET (0.03 sec)

Nếu plugin không xuất hiện trong danh sách, hãy kích hoạt lại nó:

INSTALL PLUGIN mysql_native_password SONAME 'auth_socket.so';

Table of Contents