Multi Level with dist not working as expected

[yuraxdrumz]

Hey, great project!
I have 2 nodes running with multilevel, each has local cache + dist and Redis as a backup.
Lets say one node goes down and then back up, from my understanding it is supposed to check local cache which is empty so it goes to distributed and should take the value from the 2nd node without going to Redis, but in my situation I see it goes straight to Redis and skips dist. Maybe I misunderstood the usage?

defmodule SimpleServer.Cache do
  use Nebulex.Cache,
    otp_app: :simple_server,
    adapter: Nebulex.Adapters.Multilevel

  defmodule L1 do
    use Nebulex.Cache,
      otp_app: :simple_server,
      adapter: Nebulex.Adapters.Local
  end

  defmodule L2 do
    use Nebulex.Cache,
      otp_app: :simple_server,
      adapter: Nebulex.Adapters.Dist
  end

  defmodule Primary do
    use Nebulex.Cache,
      otp_app: :simple_server,
      adapter: Nebulex.Adapters.Local
  end

  defmodule Redis do
    use Nebulex.Cache,
      otp_app: :simple_server,
      adapter: NebulexRedisAdapter
  end
end

configuration:

import Config

# Distributed Cache
config :simple_server, SimpleServer.Cache,
  levels: [
    SimpleServer.Cache.L1,
    SimpleServer.Cache.L2,
    SimpleServer.Cache.Redis
  ]

config :simple_server, SimpleServer.Cache.L1,
  # 1 hour
  gc_interval: 60 * 60

# Internal local cache used by PartitionedCache.Dist
config :simple_server, SimpleServer.Cache.L2,
    local: SimpleServer.Cache.Primary

config :simple_server, SimpleServer.Cache.Redis,
  conn_opts: [
    # Redix options
    host: "127.0.0.1",
    port: 6379
  ]

config :simple_server,
  port: 4000,
  nodes: [:"gw2@yuri-pc", :"gw1@yuri-pc"]

[cabol]

Hey, great project!

Thank you so much 😄

I have 2 nodes running with multilevel, each has local cache + dist and Redis as a backup.
Lets say one node goes down and then back up, from my understanding it is supposed to check local cache which is empty so it goes to distributed and should take the value from the 2nd node without going to Redis, but in my situation I see it goes straight to Redis and skips dist. Maybe I misunderstood the usage?

The distributed adapter implements a sharding distribution model but without replications, therefore, if the node that went down had the entry, that one won't exist anymore in the distributed cache, so it has to be retrieved from L3 (Redis). In other words, suppose we set the key "foo" and that one is stored in all levels for the first time, and the distributed cache according to the hash algorithm puts the entry in Node2. If Node 2 goes down, the entry won't exist anymore in the distributed cache (due to the sharding model, only one nose has the key, keys are distributed uniformly across the cluster but in this case, there is no replication). Does it make sense? is that your case? Please confirm and let me know, stay tuned!

[yuraxdrumz]

Thanks for the quick response!
Yep, I guess I missed the part where it is not replicated, of course it makes sense. I just tested placing random keys and restarting the node, it had only several of them, which explains the sharded but not replicated. Works like a charm, thanks a lot!

[cabol]

Excellent 😄 ... The mnesia adapter will be implemented very soon, and that one will provide full replicated topology. Furthermore, I have in the roadmap implement some kind of replication for the currently distributed adapter (sharding + replication factor).

[yuraxdrumz]

Awsome. I would love to contribute, although I am new to Elixir :)

[cabol]

It would be great, any contribution is welcomed!