30th October 2018 – KAFKA UPDATE & MULTIPLE NODE DEPLOYMENT PLAN
Congratulations to all DDKOIN users who have completed PASSPHRASE and have also completed voting for the first, second and third week.
Here are the latest DDKOIN statistics:
- Total Pre-order DNC migrated to DDKOIN : 177,651 address ( Can be verify from this link : https://explorer.ddkoin.com/#/user-i…40344646463126
- Total SUCCESS claimed PASSPHRASE : 143,322 address . 80% claimed rate.
- Total SUCCESS voted TDALLIANCE1 : 113,939 address (Can be verify from this link : https://explorer.ddkoin.com/#/delega…a4a042ec588af9 )
- Total SUCCESS voted TDALLIANCE2 : 48,813 address (Can be verify from this link : https://explorer.ddkoin.com/#/delega…45b6b8022833e5 )
- Total SUCCESS voted TDALLIANCE3 : 11,254 address (Can verify from this link : https://explorer.ddkoin.com/#/delega…42f4d443e49806 )
Kafka Development Update and Progress :
October 23th :
- Explored different Kafka modules for NodeJS client to read data in chunks.
- We found an expected solution and tried with a simple POC.
- Replaced existing producer according to the new module.
- Replaced existing consumer according to the new module.
- Checking data consumption at the consumer end.
October 24th :
- Changed the existing queue of the transactions with Kafka.
- Created 3 topics separately for queued/bundled/multisignature transactions respectively and implemented into DDK platform.
- Test Kafka implementation on two scenarios:
- Set limit 10 transactions to read from Kafka server per 10 seconds. (Not working still missing delegate slot)
- Set limit 10 transactions to read from Kafka server per 20 seconds. (Not working still missing delegate slot)
- Checking a new approach to handle Kafka queue with:
- 6 When the queue is big i.e(100 lengths), we will stop consuming message from Kafka server
- 6 When the queue is empty or less than i.e(100 lengths), We will resume consuming message from Kafka server.
October 25th :
- Created a new TOPIC for unconfirmed transactions.
- Checked for pause/resume feature in Kafka. Pause when the queue is high and resume when the queue is empty.
- Implemented new logic to consume message from Kafka when all queues are empty instead of reading on a time interval.
- Fixed an issue as duplicate transactions are sent to the Kafka server when transaction rate is high.
- Designing Kafka structure for DDKOIN is done on LOCALHOST.
October 26th :
There is a good news as Kafka implementation is quite helpful for generating blocks smoothly. However, many functionalities have been getting affected because of its implementation. Developers are checking and fixing on the module and dependencies.
We will test it on testnet first after fixing issues before making it live on mainnet.
October 29th :
1. Developers have implemented Kafka and tested on localhost. We believe, we should test it on testnet first before making it live on mainnet.
2. QA Developer will run “load testing” using available tools on Testnet.
3. We already informed DEV-OPS to setup DB replica on Mainnet so that we can merge the code on Testnet for testing Kafka transaction with some BETA tester users.
4. Kafka tested again on this day at LOCALHOST over 3 scenario 500/2, 500/10, 1000/10 use cases at a time and doing SEND & STAKE. (users/per_second, i.e : 500/2)
October 30th (TODAY) :
1. By today, we will merge Mainnet DB replica into Testnet. Devops also has made some changes regarding db replica to implement Kafka structure on our AWS server structure.
2. We will do lots of heavy testing for transaction using Kafka in TESTNET and looking for reports on LOG FILES.
October 31th (TOMORROW) :
1. We will implement Kafka on DDK Mainnet MasterNode on 31st October.
2. All users is allowable to do SEND when Kafka is implemented, please be reminded that DDK Developers will continuously monitor and keep automatic snapshot time 30 minutes if Blockchain re-building or any error involved.
Multiple Node Development Update & Progress :
October Multiple Node DEMO 26th :
1) Done install on demo server 3 Nodes with Testnet DDK blockchain network
2) Go to first Nodes UI and make first Vote, change location into second Node and check that transaction is synchronize.
3) After that, first Node will be turn off manually
4) Go to third Node and make second Vote. After second Vote we demonstrate that Reward is Done ( for DEMO multiple node we will put only 2 Vote for Staking Reward)
5) Go to second Node and make two more Votes and demonstrate Unstake DDK as transform to liquid DDK.
6) Turn on first Node and demonstrate that it’s comeback into the network and synchronize all transactions.
7) Tested on DEMO when we turn off one of node for one day and after that it comeback into the network again with all transactions. (Our result to catch the current height – one night off the node (400 blocks ) – about 4 second success on blockchain rebuilding and sync back to the current height)
October 30th – October 24th :
1) All codes for MULTIPLE NODE for issue and bug fixed will be publish and track from here : https://trello.com/b/T9gQq9ep/ddk-bugs (This Trello management will be public when MULTIPLE NODE is launch)
2) All development of MUTIPLE NODE and development of DECENTRALIZE and DISTRIBUTED DDKOIN application can be monitor from here : https://trello.com/b/GLbkM3R1/produc…n-distribution . (This Trello management will be public when MULTIPLE NODE is launch)
3) For Listing on Simex we will add RPC methods in API for integration ( we will finalise on this week in parallel ). So when multiple Node will be ready we will have all we need. Trello board for RPC application : https://trello.com/b/dv5JxWeM/product-ddk-rpc (This Trello management will be public when MULTIPLE NODE is launch).
4) When we will finish Multiple Node TESTING, we take all info about transactions from Mainnet (MasterNode) put it into multiple Node and synchronize all the network. We will use Mainnet MasterNode UI and we need make testing from both sides so that we will have decentralize network of DDKOIN using multiple node deployment.
Below is the DIAGRAM on Multiple Node mechanism for “Stake Order” transaction that would be running on decentralize transactions and assets :
Below is DIAGRAM for VOTING with REWARDS and UNSTAKE :
Multiple Node development on Trello :
Please refer BELOW for the TIMELINE and PLANNING :