Archive for January, 2023

Fixed streaming of messages to tokengraph

January 21, 2023

Today I fixed streaming of messages without needing to alter websocket clients for the protongraph-provider.

Basically I altered things slightly in my main send call:

int WorkerClient::send_to_protongraph_worker(const char* buf, int len) {
    if (ws == NULL) {
        ws = WebSocket::from_url("ws://" + std::string(pw_server_addr) + ":" + std::to_string(pw_server_port));
        if(!ws) {
            printf("Failed to connect to protongraph worker\n");
            // TODO: log error
            return 0;
        } else {
            printf("Connected to protongraph worker\n");
        }
    }
    // Wait for the connection to be established
    while (ws->getReadyState() == WebSocket::CONNECTING) {
        sleep(100);
    }
    // Add to the buffer
    ws->send(buf);
    // Set up send/receive I/O
    ws->poll();
    // Flush the buffer
    ws->dispatch(handle_message);
    delete ws;
    ws = NULL;
    return 0;
}

The trick was to remove a check on ws->getReadyState() != WebSocket::CLOSED and instead wait for getReadyState to not return state CONNECTING. That was basically the trick.

I also pulled out the setup of the WorkerClient initialisation to a bit closer to the root of the main function for the process.

With these changes I was able to achieve my objective of getting streaming working the way I wanted.

Streaming procgen messages reprise

January 20, 2023

Today I realised, upon examination of tokengraph-redis keys, that the client was only sending one packet through to tokengraph. So I wasn’t streaming the data! Looking more closely I saw a couple of spots where I had set the packet size to be quite large, 64 * 1024 – 200, which is a fair sight bigger than the 600 I had set elsewhere.

Reducing packet size emitted from the client to 600 however exposed another problem — it no longer seems that the procgen pipeline is working! So I should try to get to the bottom of that. I’ve decided among other things to build Godot 3.5 with tools from code to help debug, hopefully I can figure out how to get this working again soon.

Then I can finally fix the memory leak issue for tokengraph, before moving to tackle the much larger bugbear of memory leaks in comms.

Looks like I have an issue with sending packets to tokengraph from the protongraph-provider, I only send through one packet and then all the rest fail … looks like I might need to rip out easywsclient and put in a new library ahead of schedule.

Redis connection working

January 19, 2023

Turns out that the port for the docker network is the internal port, not the external one!

I managed to get the Redis connection working today, and also managed to set and get keys for the redis container.

✗ ./scripts/start.sh
Godot Engine v3.5.stable.official.991bb6ac7 - https://godotengine.org
Godot Engine v3.5.stable.official.991bb6ac7 - https://godotengine.org
Godot Engine v3.5.stable.official.991bb6ac7 - https://godotengine.org
OpenGL ES 2.0 Renderer: llvmpipe (LLVM 12.0.0, 256 bits)

Reading variables from config file.

Domain secured, setting secrets.
Using SSL, now setting SSL secrets.
user://config.json doesn't exists, creating the config file
user://config.json already exists, loading config
HiRedis::HiRedis(), i.e., initialising Redis.
Successfully connected to Redis server.
tokengraph-redis> keys *
1) "166107958688d98a1d-b75b-4f23-81ca-3d276a65d254:0"
2) "166107958688d98a1d-b75b-4f23-81ca-3d276a65d254"

tokengraph-redis> get "166107958688d98a1d-b75b-4f23-81ca-3d276a65d254:0"
"{\"metadata\":{\"procgenGraphId\":\"Fence\", ...,\"resources\":[]}]}}"

A great success!

Still to-do is to make sure that the keys are stored in chunks properly, and ensure that I finally get rid of that dictionary from Tokengraph so that I’m no longer using an in-memory datastore to keep track of job-related information.

Redis: connection refused

January 18, 2023

Today I:

  • Created a new docker network, “tokengraph”,
  • Improved my generated redis container tooling slightly,
  • Created a new redis service in the docker network and arranged for Tokengraph to talk to it.

However:

>./scripts/start.sh
Godot Engine v3.5.stable.official.991bb6ac7 - https://godotengine.org
Godot Engine v3.5.stable.official.991bb6ac7 - https://godotengine.org
Godot Engine v3.5.stable.official.991bb6ac7 - https://godotengine.org
OpenGL ES 2.0 Renderer: llvmpipe (LLVM 12.0.0, 256 bits)

Reading variables from config file.

Domain secured, setting secrets.
Using SSL, now setting SSL secrets.
user://config.json doesn't exists, creating the config file
user://config.json already exists, loading config
HiRedis::HiRedis(), i.e., initialising Redis.
Error: Connection refused
Error: could not connect to Redis server.

Maybe this is incorrect in hiredis?

pw_redisContext = redisConnect("tokengraph-redis", 6380);

The first argument is supposed to be an ip address. I wonder if there is some way to solve this?

Tidying up Hiredis

January 17, 2023

Today I:

  • Made sure that libssl and libssl-dev were installed within the linux image,
  • Appended ‘ssl’ to the LIBS in SConstruct for hiredis,
  • Ensured that the *.c files were appended to ‘sources’ within SConstruct for hiredis.

I managed to dealt with all compilation warnings, but on actual invocation of the exposed method …

/scripts/start.sh
Godot Engine v3.5.stable.official.991bb6ac7 - https://godotengine.org
Godot Engine v3.5.stable.official.991bb6ac7 - https://godotengine.org
Godot Engine v3.5.stable.official.991bb6ac7 - https://godotengine.org
OpenGL ES 2.0 Renderer: llvmpipe (LLVM 12.0.0, 256 bits)

Reading variables from config file.

#> information from client inbound ...

Segmentation fault

Going to require some more investigation!

A bit more debugging later, and making sure that I wasn’t calling a method on an empty variable (which I believe was the underlying reason for the segfault) …

✗ ./scripts/start.sh
Godot Engine v3.5.stable.official.991bb6ac7 - https://godotengine.org
Godot Engine v3.5.stable.official.991bb6ac7 - https://godotengine.org
Godot Engine v3.5.stable.official.991bb6ac7 - https://godotengine.org
OpenGL ES 2.0 Renderer: llvmpipe (LLVM 12.0.0, 256 bits)

Reading variables from config file.

Domain secured, setting secrets.
Using SSL, now setting SSL secrets.
user://config.json doesn't exists, creating the config file
user://config.json already exists, loading config
HiRedis::HiRedis(), i.e., initialising Redis.
Error: Connection refused
Error: could not connect to Redis server.

Bingo, so on initialise I can’t connect to the Redis server in the first place. Well certainly Tokengraph is running within a container, and my current Redis (for comms) is running in a separate container exposed to the host at localhost:6379. The tokengraph process is trying to connect to localhost:6379, but that is within the context of the container, not the context of the host machine (in this case, my laptop).

I have a couple of options. Option the first is that I simply run a separate Redis process inside the same container as Tokengraph, then things should work. Option the second that I figure out how to connect to a process running outside the container properly. I think the latter option is perhaps preferable.

Probably I need to use something like a bridge docker network? I’ll need a separate Redis process if going down this path anyway, but I think having it in the same container as Tokengraph is wrong.

More finicking with hiredis

January 16, 2023

Today I:

  • Made an abortive attempt of adding redis-plus-plus before backtracking,
  • Fixed copy of the dylib into the osx app,
  • Fixed compilation of hiredis bindings,
  • Copied a few files across to the docker image that are required for hiredis to work.

Some progress today, hopefully I can get Tokengraph writing and reading from Redis tomorrow.

Attempted to use hiredis bindings

January 15, 2023

Today I attempted to use hiredis in tokengraph, but found that there were issues loading the library.

I fixed a few bugs but I believe that there is a fundamental issue in that the SConstruct code I’m using is C++ centric, and hiredis is C. I’ll continue to look into this tomorrow, but I might have to use that wrapper library https://github.com/sewenew/redis-plus-plus after all. It certainly seems to be the pattern with librdkafka, so I guess I could copy that pattern.

Added basic methods for hiredis

January 14, 2023

Today I configured more of the boilerplate and added a few basic methods, most notably set and get, for hiredis.

Tomorrow I’ll aim to invoke these methods in the frontend to test against the default redis container before looking to run a separate dedicated redis process for tokengraph.

Added hiredis to Tokengraph

January 13, 2023

Today I added hiredis to Tokengraph (per this hiredis commit) and managed to get the .so and .dylib versions of the library compiling. I also upgraded the gcc build image from Godot 3.4.2 to Godot 3.5.

Tomorrow I intend to make a start on writing functions leveraging godot-cpp to expose hiredis get and set methods to the frontend.

Fixed several bugs

January 12, 2023

Today I fixed several issues. These were:

  • Indicate when the leaveInstanceRoute is complete. This allows the client to know when to trigger a fetchInstances request to populate the instances which are available to be joined, which is helpful when returning to the instance list page in the world management ui from an instance.
  • Only show the join page on return to the world management ui. (Previously I was showing two pages at the same time, the instance list page and the join page, which looked a bit messy).
  • Reset sessionData to default on log out. This wasn’t happening before, which meant among other things that after logout, I was still fetching a list of instances from the earlier logged in organisation id, which was not intentional.
  • Handle login and logout properly. In particular I fixed a couple of issues with user labels not being cleared and/or logout / login buttons not reflecting the appropriate state of whether one was logged in or not.

Good progress today. Tomorrow I’ll start looking at adding Redis to Tokengraph.