Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server: do not report ["encountered error"] [error=EOF] for TCPConn alive detectors #31063

Closed
bb7133 opened this issue Dec 27, 2021 · 0 comments · Fixed by #31081
Closed

server: do not report ["encountered error"] [error=EOF] for TCPConn alive detectors #31063

bb7133 opened this issue Dec 27, 2021 · 0 comments · Fixed by #31081
Assignees
Labels
affects-5.3 This bug affects 5.3.x versions. affects-5.4 This bug affects the 5.4.x(LTS) versions. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. needs-cherry-pick-release-5.4 Should cherry pick this PR to release-5.4 branch. sig/sql-infra SIG: SQL Infra type/enhancement The issue or PR belongs to an enhancement.

Comments

@bb7133
Copy link
Member

bb7133 commented Dec 27, 2021

Enhancement

After #27962, TiDB reports the handshake error if the connection is not successfully established, as a trivial bugfix. This change is released with v5.3.0

However, we found that some load balancers use the TCP connection for alive detectives, for example, the GCP and https://asktug.com/t/topic/273508.

We can easily reproduce the following error logs from TiDB:

#!/usr/bin/env python    
    
import socket    
    
TCP_IP = '127.0.0.1'    
TCP_PORT = 4000    
BUFFER_SIZE = 1024    
MESSAGE = ""    
    
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)    
s.connect((TCP_IP, TCP_PORT))    
s.close()    

You can see the following error logs from TiDB:

[ERROR] [terror.go:307] ["encountered error"] [error=EOF] [stack="github.com/pingcap/tidb/parser/terror.Log\n\t/home/bb7133/Projects/gopath/src/github.com/pingcap/tidb/parser/terror/terror.go:307\ngithub.com/pingcap/tidb/server.(*Server).onConn\n\t/home/bb7133/Projects/gopath/src/github.com/pingcap/tidb/server/server.go:519"]

The although harmless, the logs can be confusing to the users, so I think it's better to change the log level to 'DEBUG' when the io.EOF error is encountered:

// Some keep alive services will send request to TiDB and disconnect immediately.

BTW, MySQL reports the following logs for the case above:

2021-12-27T14:03:16.793444Z 12 [Note] Got an error reading communication packets

So the failed connection logs happened for MySQL, but it is better in Note level.

@bb7133 bb7133 added type/enhancement The issue or PR belongs to an enhancement. sig/sql-infra SIG: SQL Infra help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. labels Dec 27, 2021
rebelice pushed a commit to TiInterstellar/tidb that referenced this issue Jan 5, 2022
* topsql: make topsql enable only be controlled by pub/sub sink (pingcap#31209)

* ddl: support batch create table  (pingcap#28763)

* executor: fix data race in IndexMergeReaderExec (pingcap#31230)

close pingcap#31229

* server: filter the EOF error for normal closed at handshake  (pingcap#31081)

close pingcap#31063

* expression: change date add function return type (pingcap#28133)

close pingcap#27573

* support create interval partition

Signed-off-by: crazycs520 <[email protected]>

* support create interval partition (support int/timestamp partition key)

Signed-off-by: crazycs520 <[email protected]>

* parser: support alter table partitions move engine statement

Signed-off-by: crazycs520 <[email protected]>

* support ddl operation

Signed-off-by: crazycs520 <[email protected]>

* support interval partition manager

Signed-off-by: crazycs520 <[email protected]>

* support interval partition manager handle job framwork

Signed-off-by: crazycs520 <[email protected]>

* support auto create interval partition when insert meet no partition suitable error

Signed-off-by: crazycs520 <[email protected]>

* fix bug

Signed-off-by: crazycs520 <[email protected]>

* fix cancel job and load old job then continue to do

Signed-off-by: crazycs520 <[email protected]>

* make partition readonly work(not allow to insert/update/delete)

Signed-off-by: crazycs520 <[email protected]>

* add begin,end time in tables

Signed-off-by: crazycs520 <[email protected]>

* tiny fix for auto create interval partition in concurrent case

Signed-off-by: crazycs520 <[email protected]>

* init

Signed-off-by: crazycs520 <[email protected]>

* init

Signed-off-by: crazycs520 <[email protected]>

* todo: remove flag

Signed-off-by: crazycs520 <[email protected]>

* fix dumpling

Signed-off-by: crazycs520 <[email protected]>

* remove data in aws s3 when drop/truncate table/partition

Signed-off-by: crazycs520 <[email protected]>

* make hello world work

Signed-off-by: crazycs520 <[email protected]>

* remove debug info

Signed-off-by: crazycs520 <[email protected]>

Co-authored-by: xhe <[email protected]>
Co-authored-by: guo-shaoge <[email protected]>
Co-authored-by: knull-cn <[email protected]>
Co-authored-by: Meng Xin <[email protected]>
@tiancaiamao tiancaiamao added affects-5.3 This bug affects 5.3.x versions. needs-cherry-pick-release-5.4 Should cherry pick this PR to release-5.4 branch. labels Jan 7, 2022
@VelocityLight VelocityLight added the affects-5.4 This bug affects the 5.4.x(LTS) versions. label Jan 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-5.3 This bug affects 5.3.x versions. affects-5.4 This bug affects the 5.4.x(LTS) versions. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. needs-cherry-pick-release-5.4 Should cherry pick this PR to release-5.4 branch. sig/sql-infra SIG: SQL Infra type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants