-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[hbase10] Address #701 by mimicking the same locks from the HBase 0.9… #1028
Conversation
…he HBase 0.98 client in the HBase 10 client.
@@ -82,7 +84,7 @@ | |||
* @See #CONNECTION_LOCK. | |||
*/ | |||
private static Connection connection = null; | |||
private static final Object CONNECTION_LOCK = new Object(); | |||
//private static final Object CONNECTION_LOCK = new Object(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just delete the line. Don't comment out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
doh, thanks.
@@ -225,7 +224,7 @@ public void cleanup() throws DBException { | |||
|
|||
public void getHTable(String table) throws IOException { | |||
final TableName tName = TableName.valueOf(table); | |||
synchronized (CONNECTION_LOCK) { | |||
synchronized (TABLE_LOCK) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#701 asked for this lock to be removed, if I am reading it correctly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
He did mention it but I need confirmation from an HBase dev that the table fetch is thread safe in 1.X. I think it wasn't in the past.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Connection objects in hbase 1.y are threadsafe, so getTable is fine. The returned Table object is not threadsafe, but making them is supposed to be lightweight and done per-thread when needed.
int threadCount = THREAD_COUNT.decrementAndGet(); | ||
if (threadCount <= 0 && connection != null) { | ||
connection.close(); | ||
connection = null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This synchronized block should not be needed. The closing of the connection is protected by the decrement of the atomic integer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, yanked the sync.
@@ -145,8 +147,8 @@ public void init() throws DBException { | |||
} | |||
|
|||
try { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we merge this try into the try after the "Terminate right now" comment?
@@ -179,7 +181,7 @@ public void init() throws DBException { | |||
String table = getProperties().getProperty(TABLENAME_PROPERTY, TABLENAME_PROPERTY_DEFAULT); | |||
try { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you merge the try above into this try you would have something like:
if( THREAD_COUNT.getAndIncrement() == 0) {
String table = getProperties().getProperty(TABLENAME_PROPERTY, TABLENAME_PROPERTY_DEFAULT);
try {
if (connection == null) {
// Initialize if not set up already.
connection = ConnectionFactory.createConnection(config);
// Validate the table name.
final TableName tName = TableName.valueOf(table);
connection.getTable(tName).getTableDescriptor();
}
} catch (java.io.IOException e) {
throw new DBException(e);
}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, merged em.
@@ -225,7 +224,7 @@ public void cleanup() throws DBException { | |||
|
|||
public void getHTable(String table) throws IOException { | |||
final TableName tName = TableName.valueOf(table); | |||
synchronized (CONNECTION_LOCK) { | |||
synchronized (TABLE_LOCK) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Connection objects in hbase 1.y are threadsafe, so getTable is fine. The returned Table object is not threadsafe, but making them is supposed to be lightweight and done per-thread when needed.
Yanked the lock so we should be good now @busbey, thanks! |
} | ||
} | ||
int threadCount = THREAD_COUNT.decrementAndGet(); | ||
if (threadCount <= 0 && connection != null) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this needs to be in a lock that covers checking if it's null and setting it to null, othewise we race the initialization above that only creates a new connection when it's null.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But Alan was right in that this should only fire when the atomic int returns a 0 at which point only one thread will perform the null check and close + null the connection.
Or are you thinking of the case where there may be some initialization issue wherein a run initializes multiple threads but the initializations fail and possibly cleanup is executed on a thread before initialization begins in another thread?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right exactly.
…8 client in the HBase 10 client.