Skip to content

Commit

Permalink
[master] fix race between shutdown and TxnManager init
Browse files Browse the repository at this point in the history
I saw the following TSAN report while running one of the newer tests
introduced recently (TxnOpDispatcherITest.LifecycleBasic).  As I can
see, the race isn't related to the TxnOpDispatcher itself, but rather
to the way how Master::state_ field is used in the master's code.

WARNING: ThreadSanitizer: data race (pid=8313)
  Read of size 4 at 0x7b78000006b0 by thread T165:
    #0 kudu::master::Master::InitTxnManagerTask() src/kudu/master/master.cc:312:9 (libmaster.so+0x26f28b)
    #1 kudu::master::Master::ScheduleTxnManagerInit()::$_1::operator()() const src/kudu/master/master.cc:299:46 (libmaster.so+0x2741a1)

  Previous write of size 4 at 0x7b78000006b0 by main thread:
    #0 kudu::master::Master::ShutdownImpl() src/kudu/master/master.cc:406:12 (libmaster.so+0x26da08)
    #1 kudu::master::Master::Shutdown() src/kudu/master/master.h:74:5 (libmaster.so+0x275d49)
    #2 kudu::master::MiniMaster::Shutdown() src/kudu/master/mini_master.cc:117:14 (libmaster.so+0x2a9c38)
    #3 kudu::cluster::InternalMiniCluster::ShutdownNodes(kudu::cluster::ClusterNodes) src/kudu/mini-cluster/internal_mini_cluster.cc:230:22 (libmini_cluster.so+0x84037)
    #4 kudu::cluster::MiniCluster::Shutdown() src/kudu/mini-cluster/mini_cluster.h:79:5 (libitest_util.so+0xa59d4)
    #5 kudu::cluster::InternalMiniCluster::~InternalMiniCluster() src/kudu/mini-cluster/internal_mini_cluster.cc:94:3 (libmini_cluster.so+0x82a03)

This patch fixes the issue by making Master::state_ atomic.

Change-Id: Ifa7541aa7dc7dbdb8e6af5c1f40edf23b850fc92
Reviewed-on: http://gerrit.cloudera.org:8080/17076
Tested-by: Kudu Jenkins
Reviewed-by: Andrew Wong <[email protected]>
  • Loading branch information
alexeyserbin committed Feb 17, 2021
1 parent f24ec6a commit 4a88bde
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/kudu/master/master.h
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ class Master : public kserver::KuduServer {
kStopping,
};

MasterState state_;
std::atomic<MasterState> state_;

std::unique_ptr<MasterCertAuthority> cert_authority_;
std::unique_ptr<security::TokenSigner> token_signer_;
Expand Down

0 comments on commit 4a88bde

Please sign in to comment.