Implement georadiusbymember

[romange]

see here: https://redis.io/commands/georadiusbymember/

[binjamil]

From the Redis docs:

As of Redis version 6.2.0, this command is regarded as deprecated.

It can be replaced by GEOSEARCH and GEOSEARCHSTORE with the BYRADIUS and FROMMEMBER arguments when migrating or writing new code.

Why there is a need to support a deprecated command?

[romange]

1. Because of the existing code which most likely has not fully aligned with newest versions of Redis.
2. The newer command is more complicated so georadiusbymember is the convenient milestone before implementing that

[binjamil]

I'm interested to work on this but having trouble building dragonfly locally. I get the following error when I configure the build. I've verified that all dependencies are installed, including Boost, on my Linux dist openSUSE Tumbleweed (Version 20230501).

CMake Error at /usr/lib64/cmake/Boost-1.82.0/BoostConfig.cmake:141 (find_package):
  Could not find a package configuration file provided by "boost_context"
  (requested version 1.82.0) with any of the following names:

    boost_contextConfig.cmake
    boost_context-config.cmake

  Add the installation prefix of "boost_context" to CMAKE_PREFIX_PATH or set
  "boost_context_DIR" to a directory containing one of the above files.  If
  "boost_context" provides a separate development package or SDK, be sure it
  has been installed.
Call Stack (most recent call first):
  /usr/lib64/cmake/Boost-1.82.0/BoostConfig.cmake:262 (boost_find_component)
  /usr/share/cmake/Modules/FindBoost.cmake:594 (find_package)
  helio/cmake/third_party.cmake:233 (find_package)
  CMakeLists.txt:29 (include)


-- Configuring incomplete, errors occurred!

Perhaps Boost v1.82.0 might not be compatible. Should I use a different version for Boost or is this due to something else?

[chakaz]

Where is boost installed? You may need to add relevant include paths and link paths when you invoke cmake

[romange]

@binjamil how did you install boost or how did you verify it's installed?

[binjamil]

I installed it using the package manager for suse, which is zypper.

I can see the libraries in /lib64

[chakaz]

I don't see libboost_context there..?

[romange]

Right. try running zypper se libboost*devel

[binjamil]

Turns out that following three Boost components were not installed, so I had to manually install them as well:

• libboost_context-devel
• libboost_system-devel
• libboost_fiber-devel

This fixed the configuration but I get the following linking error when I run ninja dragonfly:

/usr/lib64/gcc/x86_64-suse-linux/13/../../../../x86_64-suse-linux/bin/ld: cannot find third_party/libs/reflex/lib/libreflex.a: No such file or directory
/usr/lib64/gcc/x86_64-suse-linux/13/../../../../x86_64-suse-linux/bin/ld: cannot find third_party/libs/gperf/lib/libprofiler.a: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

I can see the static libraries are not present in lib directory but they are inside lib64 instead:

➜  ls third_party/libs/reflex/lib64 
libreflex.a  libreflexmin.a

Any advice on how to change cmake config so the linker looks in the correct lib64 directory?

[romange]

I suggest that you manually put them where build expects them to be.

[azuredream]

Hi romange

Could you share where can I find the log output of VLOG(2) and LOG()?

After reading more about the transaction framework, I'm revisiting this feature. Last time I didn't make "store" works perfectly (It works in CLI but not in test). Now I have a guess but need to check log.

Thanks!

[chakaz]

@azuredream logs are written to log file (when Dragonfly starts it prints the lookup paths), but you could pass --logtostdout to get it to write to the console.
The output of LOG() is always printed, but in order for VLOG() to really print you'll have to pass in --vmodule=<filename>=<level>, where <filename> is the file which has the VLOG() (I think that without the .cc suffix, but I might not remember correctly), and level is some number that should be higher than or equal to the level you want (so for VLOG(2) you should pass 2 or higher)
HTH

[azuredream]

Thanks. How can I enable VLOG for multiple files? --vmodule=transaction=3 --vmodule=engine_shard_set=3 didn't work.

I enabled VLOG for transaction.cc. It appears that when all RunInShard() was performed before Remove from txq, the query would work. Otherwise would fail.
ZUNIONSTORE always works correctly. Not sure how to guarantee that order in my code.

Succeeded:
127.0.0.1:6379> GEORADIUSBYMEMBER Europe Madrid 900 KM STORE test3
(integer) 2
I20231226 19:07:48.435060 128341 transaction.cc:828] ExecuteAsync GEORADIUSBYMEMBER@10/2 (1822)
I20231226 19:07:48.435163 128341 transaction.cc:819] Wait on Exec GEORADIUSBYMEMBER@10/2 (1822)
I20231226 19:07:48.435267 128353 transaction.cc:871] PollExecCb GEORADIUSBYMEMBER@10/2 (1822) sid(12) 2
I20231226 19:07:48.435258 128352 transaction.cc:871] PollExecCb GEORADIUSBYMEMBER@10/2 (1822) sid(11) 2
I20231226 19:07:48.435364 128353 transaction.cc:451] RunInShard: GEORADIUSBYMEMBER@10/2 (1822) sid:12 13
I20231226 19:07:48.435402 128352 transaction.cc:451] RunInShard: GEORADIUSBYMEMBER@10/2 (1822) sid:11 13
I20231226 19:07:48.435443 128353 transaction.cc:511] Remove from txqGEORADIUSBYMEMBER@10/2 (1822)
I20231226 19:07:48.435487 128352 transaction.cc:511] Remove from txqGEORADIUSBYMEMBER@10/2 (1822)
I20231226 19:07:48.435495 128353 transaction.cc:897] ptr_release GEORADIUSBYMEMBER@10/2 (1822) 0
I20231226 19:07:48.435528 128352 transaction.cc:897] ptr_release GEORADIUSBYMEMBER@10/2 (1822) 0
I20231226 19:07:48.435606 128341 transaction.cc:821] Wait on Exec GEORADIUSBYMEMBER@10/2 (1822) completed

Failed:
127.0.0.1:6379> GEORADIUSBYMEMBER Europe Madrid 900 KM STORE test4
(error) ERR Member not found
I20231226 19:07:48.435739 128341 transaction.cc:828] ExecuteAsync GEORADIUSBYMEMBER@10/2 (1822)
I20231226 19:07:48.435851 128341 transaction.cc:819] Wait on Exec GEORADIUSBYMEMBER@10/2 (1822)
I20231226 19:07:48.435883 128352 transaction.cc:871] PollExecCb GEORADIUSBYMEMBER@10/2 (1822) sid(11) 2
I20231226 19:07:48.435930 128352 transaction.cc:451] RunInShard: GEORADIUSBYMEMBER@10/2 (1822) sid:11 13
I20231226 19:07:48.435948 128353 transaction.cc:871] PollExecCb GEORADIUSBYMEMBER@10/2 (1822) sid(12) 1
I20231226 19:07:48.436018 128352 transaction.cc:897] ptr_release GEORADIUSBYMEMBER@10/2 (1822) 1
I20231226 19:07:48.436046 128353 transaction.cc:451] RunInShard: GEORADIUSBYMEMBER@10/2 (1822) sid:12 13
I20231226 19:07:48.436123 128353 transaction.cc:897] ptr_release GEORADIUSBYMEMBER@10/2 (1822) 1
I20231226 19:07:48.436196 128341 transaction.cc:821] Wait on Exec GEORADIUSBYMEMBER@10/2 (1822) completed
I20231226 19:07:48.436427 128341 transaction.cc:72] Transaction GEORADIUSBYMEMBER@10/2) destroyed

[chakaz]

Thanks. How can I enable VLOG for multiple files? --vmodule=transaction=3 --vmodule=engine_shard_set=3 didn't work.

With a comma, i.e. --vmodule=transaction=3,engine_shard_set=3

Re/ the issue you're seeing: I'm not sure what changes you've made (didn't see your code), but if I had to guess I'd say that perhaps you're trying to write to the destination key from the same callback that reads the source key(s)?

If so, you shouldn't do that, as they may be present in different shards.

[azuredream]

Thanks! I've created a Draft PR if you have free time to take a look at my change.
There are at most 3 hops
hop1. Getting latitude longitude from member name
hop2. Getting nearby nodes around <latitude ,longitude >
hop3. write to store key if needed

This random failure only occurs when I use "STORE" option. GEORADIUSBYMEMBER Europe Madrid 900 KM always works.
_127.0.0.1:6379> GEORADIUSBYMEMBER Europe Madrid 900 KM STORE test8
(integer) 2
127.0.0.1:6379> GEORADIUSBYMEMBER Europe Madrid 900 KM STORE test8
(error) ERR Member not found1 #Madrid not found, hop1 failed
127.0.0.1:6379> GEORADIUSBYMEMBER Europe Madrid 900 KM

1. 1. "Madrid"
2. 1. "Lisbon"_

The only difference between two workflows is hop3.
However, when it fails, it fails at hop1, getting score from member name.
Both cases worked on the same shards, 10 and 12.