PHP Classes

PHP MySQL Replication to Kafka Docker: Docker container Move replication events to Kafka

Recommend this page to a friend!
     
  Info   Example   View files Files   Install with Composer Install with Composer   Download Download   Reputation   Support forum   Blog    
Ratings Unique User Downloads Download Rankings
Not yet rated by the usersTotal: 47 All time: 10,759 This week: 455Up
Version License PHP version Categories
php-mysql-replicatio 1.0.0MIT/X Consortium ...5PHP 5, Databases, Systems administration
Description 

Author

Kacper Rowinski

This package can be used to move replication events to Apache kafka.

It provides the files to setup a Docker container that runs a MySQL replication environment.

The PHP classes provided by the package can use Apache Kafta to exchange MySQL replication events between MySQL server instances.

Innovation Award
PHP Programming Innovation award nominee
February 2021
Number 7

Prize: 		Docker is an useful tool that is use to assemble containers that can run with specific software on a server.

When you need to scale an application that uses MySQL, replication is one of the possible solutions that can be used to distribute the database server load among different servers.

Using docker you can assemble a solution for creating multiple MySQL server instances.

This PHP package can be used to manage MySQL replication events in docker contains using Apache kafka.

Manuel Lemos
Picture of Kacper Rowinski
  Performance   Level  
Name: Kacper Rowinski <contact>
Classes: 14 packages by
Country: Poland Poland
Age: 40
All time rank: 91426 in Poland Poland
Week rank: 215 Up2 in Poland Poland Up

Level 4 StarStarStarStar

 Innovation award
Innovation award
Nominee: 8x

Recommendations

I need a PHP class for perform tally data synchronization
Tally data synchronization service

Example

#!/usr/bin/env php
<?php

declare(strict_types=1);

require __DIR__ . '/../vendor/autoload.php';

use ReplicationToKafka\ReplicationToKafkaCommand;
use Symfony\Component\Console\Application;

$application = new Application();
$application->add(new ReplicationToKafkaCommand());
/** @noinspection PhpUnhandledExceptionInspection */
$application->run();

Details

php-mysql-replication-to-kafka-docker

Moves mysql replication events to kafka docker.

Env:

REPLICATOR_DEBUG: "false"

print worker action to log

REPLICATOR_USER: "root"

your mysql user (mandatory)

REPLICATOR_HOST: "r2k-mysql"

your mysql host/ip (mandatory)

REPLICATOR_PASSWORD: "root"

your mysql password (mandatory)

REPLICATOR_PORT: "3306"

our mysql host port (default 3306)

REPLICATOR_SLAVE_ID: "100"

script slave id for identification (SHOW SLAVE HOSTS)

REPLICATOR_START_FROM_GTID: ""

GTID marker(s) to start from

REPLICATOR_START_BIN_LOG_FILE_NAME: ""

bin log file name to start from

REPLICATOR_START_BIN_LOG_POSITION: "0"

bin log position to start from

REPLICATOR_START_FROM_MARIA_DB_GTID: ""

MariaDB GTID marker(s) to start from

REPLICATOR_TABLE_CACHE: "1024"

some data are collected from information schema, this data is cached.

REPLICATOR_HEARTBEAT: "0.0"

sets the interval in seconds between replication heartbeats.

REPLICATOR_LISTEN_ON_TABLES: "foo,bar"

only listen on given tables (default all tables)

REPLICATOR_LISTEN_ON_DATABASES: "foo,bar"

only listen on given databases (default all tables)

`REPLICATOR_LISTEN_EVENT_IDS: "73,32"

only listen for given events, look for ids in MySQLReplication\Definitions\ConstEventType::class`

REPLICATOR_IGNORE_EVENT_IDS: "73,32"

ignore given events, look for ids in MySQLReplication\Definitions\ConstEventType::class`

KAFKA_DEBUG: "false"

shows rdkafka events

KAFKA_BROKER: "r2k-kafka:9092"

brokers host:port

KAFKA_TOPIC: "test"

topic name to publish mysql events

KAFKA_RETRIES: "3"

how many tries worker should have before giving up and throw exception

KAFKA_TIMEOUT: "10000"

rdkafa flush timeout ms

Also, you can look and docker-compose.yml in repo.


  Files folder image Files (12)  
File Role Description
Files folder image.github (1 directory)
Files folder imagebin (1 file)
Files folder imagesrc (4 files)
Accessible without login Plain text file composer.json Data Auxiliary data
Accessible without login Plain text file composer.lock Data Auxiliary data
Accessible without login Plain text file docker-compose.yml Data Auxiliary data
Accessible without login Plain text file Dockerfile Data Auxiliary data
Accessible without login Plain text file LICENSE Lic. License text
Accessible without login Plain text file README.md Doc. Documentation

  Files folder image Files (12)  /  .github  
File Role Description
Files folder imageworkflows (1 file)

  Files folder image Files (12)  /  .github  /  workflows  
File Role Description
  Accessible without login Plain text file docker-publish.yml Data Auxiliary data

  Files folder image Files (12)  /  bin  
File Role Description
  Accessible without login Plain text file application.php Example Example script

  Files folder image Files (12)  /  src  
File Role Description
  Plain text file KafkaSubscriber.php Class Class source
  Plain text file ReplicationToKafkaCommand.php Class Class source
  Plain text file ReplicatorResume.php Class Class source
  Plain text file Tools.php Class Class source

The PHP Classes site has supported package installation using the Composer tool since 2013, as you may verify by reading this instructions page.
Install with Composer Install with Composer
Downloadphp-mysql-replicatio-2021-02-01.zip 14KB
Downloadphp-mysql-replicatio-2021-02-01.tar.gz
Install with ComposerInstall with Composer
 Version Control Unique User Downloads Download Rankings  
 100%
Total:47
This week:0
All time:10,759
This week:455Up

  Applications that use this package  
No pages of applications that use this class were specified.

Add link image If you know an application of this package, send a message to the author to add a link here.