Client as peer path implemented

April 10, 2020

I’ve implemented the client-as-peer path. However, I found out that there is a new port each time for each UDP client I create to communicate between UDP servers! Indeed, the port for server_id 41 is not the same recorded port in the db for the session owner for server id 41.

So it looks like I still have the problem of figuring out how on earth to hook two separate UDP servers together and get them talking.

Having a quick eyeball at https://en.wikipedia.org/wiki/UDP_hole_punching, it looks like the work I’ve done so far is not a complete waste, though. Progress should theoretically be possible.

In this case, then, it seems that I should maybe try to re-tool the listener as also a handshaker between different UDP servers who wish to connect and talk to one another.

Copying the approach on the page:

  1. A and B each begin a UDP conversation with S; the NAT devices NA and NB create UDP translation states and assign temporary external port numbers EPA and EPB
  2. S examines the UDP packets to get the source port used by NA and NB (the external NAT ports EPA and EPB)
  3. S passes EIPA:EPA to B and EIPB:EPB to A
  4. A sends a packet to EIPB:EPB.
  5. NA examines A’s packet and creates the following tuple in its translation table: (Source-IP-A, EPA, EIPB, EPB)
  6. B sends a packet to EIPA:EPA
  7. NB examines B’s packet and creates the following tuple in its translation table: (Source-IP-B, EPB, EIPA, EPA)
  8. Depending on the state of NA‘s translation table when B’s first packet arrives (i.e. whether the tuple (Source-IP-A, EPA, EIPB, EPB) has been created by the time of arrival of B’s first packet), B’s first packet is dropped (no entry in translation table) or passed (entry in translation table has been made).
  9. Depending on the state of NB‘s translation table when A’s first packet arrives (i.e. whether the tuple (Source-IP-B, EPB, EIPA, EPA) has been created by the time of arrival of A’s first packet), A’s first packet is dropped (no entry in translation table) or passed (entry in translation table has been made).
  10. At worst, the second packet from A reaches B; at worst the second packet from B reaches A. Holes have been “punched” in the NAT and both hosts can directly communicate.

So I’d like to:

  • First create two separate UDP servers, fabricator1 (A) and fabricator2 (B). Both of these should start a dialogue with the listener (S). i.e. I would like to create two separate “ping loops” where there is a back and forth with different UDP clients & different port numbers mediated by NA and NB between A and S, and B and S.
  • Each of A and B should register their intent to play on a particular server by id (or whether or not they want to create and own a server on a particular id).
  • The listener S then waits until at least two UDP servers in dialogue, A and B, want to connect to the same Sandbox server_id.
  • Then, the listener S should pause the loop, and then:
    • Listener S to examine the current last UDP packets to get the source port used by NA and NB (the external NAT ports EPA and EPB)
    • S to try passing EIPA:EPA to B and EIPB:EPB to A
    • A sends a packet to EIPB:EPB.
    • NA examines A’s packet and creates the following tuple in its translation table: (Source-IP-A, EPA, EIPB, EPB)
    • B sends a packet to EIPA:EPA.
    • NB examines B’s packet and creates the following tuple in its translation table: (Source-IP-B, EPB, EIPA, EPA)
  • NAT hole punching is complete.
  • S should resume the ping loop for the session owner of the relevant Sandbox server with id: server_id, in case additional clients want to join that server.

Of course, pausing the ping loop and restarting again doesn’t really scale, when say a lot of clients want to connect to a server at the same time. So ideally I’d want to spin off a separate queue or process and then process things there – if I wanted to get fancy.

But for a minimal viable product I think the above is enough.

Created session owner in db

April 9, 2020

Today I actioned steps 3 to 5 of the flow before. In short:

  • The jumpstarter tells the fabricator to talk to the listener, sending payload `var udp_payload = { message: { type: ‘create-new-server’, payload: { server_name: ‘Example Server Name’ } }, endpoint: ‘/servers.json’, source: ‘jumpstarter’ }`
  • The fabricator forwards this message on to the listener.
  • The listener posts with axios to the book-keeper at ‘/servers.json’ with payload `{ server_name: ‘Example Server Name’ }`.
  • The book-keeper responds to the listener with the server_id.
  • The listener posts to the fabricator with the response from the book-keeper.
  • The fabricator sends a new request to the listener with payload
server_id = payload["message"]["id"];
endpoint = `/servers/${server_id}/clients.json`;
message = { type: 'append-peer-to-server', payload: { player_handle: 'Fred', server_id: server_id, is_session_owner: true } };
udp_payload = { message: message, endpoint: endpoint, source: 'fabricator' };
  • The listener makes another axios post request to `/servers/:server_id/clients.json` on the book-keeper.

Now all that remains to do is to implement the flow for joining a server that is already running. I will look into that next.

Foundations for remaining steps

April 9, 2020

Today I laid some of the foundations for the remaining steps 3 to 5 of the diagram in my earlier post. I reproduce the diagram here for clarity:

Per my previous post, steps (1) and (2) are now working. Essentially I’d like now to be able to send a signal back to the Fabricator (3) and then send a new request to the Listener based on that new information (4), and then process that request in the Book-Keeper (5).

This flow will require understanding the source of a request, as well as endpoint and the contents of the message. e.g.

payload = { message: data, endpoint: '/servers/4/clients.json', source: 'fabricator' }

I’ve started working on this today, hopefully I can make a bit more progress on this over the coming days.

Jumpstarter to Book-keeper flow working

April 8, 2020

Today I managed to get communication from Jumpstarter to Fabricator, from Fabricator to Listener, and from Listener to Book-keeper (and then on to the database) working.

The next step will be to figure out how to process a response from the Book-keeper to the Listener properly, and then properly manage a dialogue between the Listener and the Fabricator in order to instantiate a new server (with a corresponding new user-as-server-host) properly.

Helper methods for Listener project

April 5, 2020

Today I extracted a number of methods out for the listener project. I have the fabricator on port 7778, and the listener on port 7777. And of course the book-keeper is on 3000. A key abstraction is that creating a throwaway enet_client is now abstracted to a call to “post_enet_request(server_address, payload)”, with payload = { message: data_object, endpoint: book_keeper_endpoint }.

The jumpstarter will initiate the fabricator, the fabricator will post_enet_request to talk to the listener, the listener will spawn an axios request to post to the book-keeper.

When the book-keeper responds, the listener will post_enet_request to the fabricator, and the fabricator will then with the server_id post_enet_request to the listener and on a different book_keeper_endpoint. At least that’s the theory, I’ll see if I can continue to nudge this along this week.

D&D and book club

April 4, 2020

Today I kicked off a new D&D campaign as GM, and also read a few more chapters of la belle sauvage.

Tomorrow hopefully I will be able to progress with some more project work, in lieu of today.

Fabricator, Listener, Book-Keeper, Jumpstarter.

April 3, 2020

Today I had another bit of a chew at the listener problem. Recall that we want to be able to send messages between the Sandbox and the Book-keeper via the Listener.

So far I have managed to send messages from the Listener to the Book-keeper, via a client from the Fabricator (which essentially is a mock for the Sandbox server). However returning messages to the Fabricator has proven problematic.

After a bit of thought, I have realised that my problem is a little bit more subtle than I originally thought; I need two ENet servers, and spawn ENet clients if/as needed to communicate between them. I guess this should not be a great surprise. But essentially I believe the situation looks like this:

Step (1) has sort of been half done; I have an ENet Client to send a create server request to the Listener. Step (2) has been completely done; I have been able to successfully send an axios post request to the servers.json endpoint of the Book-keeper running locally, and get a valid response.

So that leaves:

  • step (0): “Fabricate a Fabricator”, make sure that we have a Fabricator running as a separate ENet server on another port that is not the Listener port,
  • step (3): “L: Here is your created server”, create a client to broadcast the response from (2) back to the Fabrictor
  • step (4): “F: Thankyou, I would like to create a client on said server”, create a client to send a create client request based on the response received via step (3), and
  • step (5): “L: Very good, I will relay your request to the book-keeper”, create an axios post request to create a client on the relevant server & with the correct ownership status within the Book-Keeper

It also goes without saying that I would like conditions under which the Fabricator will send a request to create a Server to the Listener, or merely just create a client on an existing server. This might necessitate creation of yet another ENet client to jumpstart the whole process – but this would be a throwaway piece of the puzzle and therefore does not need to listen to responses. In other words, we need another step:

  • step (0b): create an ENet client (Jumpstarter) to jumpstart the Fabricator-Listener-BookKeeper discussion.

So not huge progress today, but I have a better idea as to the shape of what I need to do.

Last few days

April 2, 2020

I’ve let the last few days slip away a bit. Yesterday and to a lesser extent the day before, I was preparing to run a campaign session this Saturday, so I guess that gives me a decent excuse! And I sought to read a couple more chapters of La Belle Sauvage for a book club. I’m not making very rapid progress with that, but we will see how I fare in the days ahead. A fortnightly read is quite ambitious for me; normally I tend to take a lot more time to finish a book – that is, if I ever quite finish one!

Regardless, now that most of that preparation for Saturday’s session is (more or less) sorted, I can now return to the matter of working on the project.

First things first, I need to tidy up my terminal. Turns out that ruby-2.5.6 doesn’t work for reasons, so I switched back to ruby-2.4.1. Also, openssl is a bit broken, so I brew switch openssl 1.0.2t (switched to an older version of openssl).

Next, I need to tidy up my local development environment. The way to do that is to presumably have a different way of running in development to production. In particular, I don’t really want to mess around with puma and nginx locally; I just want to use the default rails behaviour.

Turns out the answer to that is simple, bundle exec rails s -b 0.0.0.0 , while making sure that postgres is running.

Okay, now to look into doing things!

The story that I have to look at next is “Listener server improvements for peer-as-server”. In particular, I want to:

* I need to allow the listener to take a request with payload of the form `{ type: ‘peer-as-server’ }` (bearing in mind that for the other case that I plan to action later – a client connecting to the server – I will be interested in the listener handling payload with `{ type: ‘peer-as-client’, data: { server_id: <server_id> } }`, which will be handled differently) and then read the ip and port of the requester.

* The listener should then be able to forward this data to the book-keeper at the /servers.json endpoint

* From earlier, the book-keeper should respond with `{ id, server_ip, server_port }`. The listener should take this response and itself respectively send an ENet request to the ENet requestee that triggered it reaching out to the book-keeper in the first place. (I should mock the ENet requestee).

So there are three parts to this. The first part requires some mocking of the test data, but should be achievable. Basically, I would like to run my UDP server locally, as well as the book-keeping server (which I have just figured out above), and then send a POST request to the book-keeper with { request_ip: <ip_used_by_peer_process_that_made_request_to_listener>, request_port: <port_used_by_peer_process_that_made_request_to_listener>, is_server: true }.

Looks like a trip down memory lane is warranted; digging up the code for the listener:

  • Run node server.js first to run the server, that decides to do things with incoming traffic. “server.js” here is the Listener.
  • Next, bers -b 0.0.0.0 in the Book-keeper locally, to make sure that that is running.
  • Finally, run node client.js second, to run the mocked test data, that will spoof a call from the ENet peer to the listener. Let’s call this the Fabricator.

If all is working as intended:

  • The Fabricator will send a call to the Listener
  • The Listener will then on-forward this to the Book-Keeper.
  • The Book-Keeper will create a record in the Postgres Database, and then respond to the Listener.
  • The Listener should then respond to the Fabricator.

Okay, so we have a plan! But now I am getting a bit tired. I’ll try to pick this up again tomorrow; maybe I can complete another chapter of that book before I tie things up for today.

Book-keeping server changes actioned

March 30, 2020

Today I actioned changes to the book-keeping server.

  • Installed ruby-2.5.6 locally, had some environment issues
  • Couldn’t test rails locally because of nginx, I need to figure out how to do that later
  • Made the changes to the book-keeping server by actioning a migration (server_ip and server_port columns on servers table).
  • `git push origin master` (github)
  • `git push heroku master` (heroku deployment)
  • `heroku run db:migration`
  • `heroku restart` (so that heroku would pick up the changes!)

I checked using postman. I was thrown for a bit because I thought that I had made all the changes that were required, but heroku didn’t seem to be registering that the columns were present. Even more puzzling was when I run heroku run rails console, Server.connection, Server and saw that the columns were there.

I finally figured out that I needed to run heroku restart for the state of the heroku app to pick up the new columns.

March retrospective

March 27, 2020

General morale

I’ll be honest, not fantastic. I think my morale was at a low ebb over the last few weeks, as I grappled with a fair amount of uncertainty and adaptation to change. And I understand that things will get worse before they get better.

I am very fortunate though, in that I have been blessed in a number of ways, and in a sense I understand that I do not deserve the grace that has been extended to me in these respects. Nonetheless, I am grateful to be able to continue my regular paid work, as it helps to distract me from what else is currently going on. I am grateful for being in a part of the city where I live where it is not too crowded, and there is good access to amenities. I am also grateful for the health of my loved ones and friends, and for the broader communities that I am part of.

My hope is that my local community response follows this pattern, and that leaders in this time of crisis are allowed to step up and take control of the situation. I also hope that this crisis will not go to waste, but will allow people to reflect on things, such as:

  • The perils of complacency
  • The importance of good government
  • The importance of community, and the things that build strong community, such as the church
  • Recognising that we are all connected
  • The understanding of the ephemerality of all things, including our lives, and that it is important to bear this in mind, particularly when considering whether to build bigger barns
  • The importance of respecting expert opinion
  • The dangers of point scoring for shallow political gain
  • The importance of “servant leadership

In terms of more utilitarian things, I was thinking that people might ask themselves:

  • Whether the old patterns of work need be set in stone. Can more of our workforce work remotely / work from home?
  • How they could seed the innovation coming from this unfolding humanitarian disaster into business improvements when the new normal finally arrives – i.e., where are the opportunities here?
  • Can we reduce our carbon footprint in particular ways, bearing in mind the much slower unfolding freight train – with more serious and lasting implications (unfolding over a 200 year time period, and with interest coming due ~ 2030 -> 2070) that is the potential degradation of the habitability of our planet?

What went well?

As in January, so in this month; over the last month I have made leaps and bounds in terms of progress on the project – even though from March 19 I essentially stopped further work for a bit.

  • I discovered the godot_voxel module and how to build procedural terrain essentially from a random seed using that module.
  • I learned a bit about godot’s structure and fooled around a bit with C++, blowing some dust out of the corners of my mind where, long ago, I learned about pointers and addresses for objects in said language
  • I made significant inroads into the Non-Authoritative Networking that I wanted to implement – in particular, I learned about how one could extract the ip of a request from nginx and other things.

Also I did not complete one of my main objectives – that was to allow clients at different ips to speak to each other, I did manage to:

  • Complete the wiring for selecting a server from a list, in-as-much as the client attempted to establish a peer connection to the peer-as-server, albeit on the wrong port. However, I did manage to get the ip correct!
  • I also understood that my naive “just use rails” for Non-Authoritative Networking was not going to work, and was able to successfully architect for an intermediate Node.js server – the Listener – to sit between the Game Client and the Book-keeping Server.

What didn’t go so well?

As mentioned above in some detail, I kind of held it together this month until about a week ago, when my sleeping patterns started to become erratic, and I just decided to focus on self-care for a bit.

However, apart from that (and the evident cause), it is worth calling out the blow-out of complexity of the networking. I do understand kind of why it did – I didn’t realise beforehand that I really wanted to implement some form of NAT punchthrough solution, and after I figured out how to extract ips from nginx after a reasonable amount of engineering effort, I discovered that that solution was not fit for purpose, as the port would shift with each request!

So, in other words, I needed a way to register the port that the server process was using from within the server process. And that meant figuring out how to sandwich a server in between the Sandbox Client and the Book-Keeper that would listen to UDP traffic etc.

The good news is that I figured out roughly how to do that, and even started building prototypes. I also created cards for the next sprint, threw out some things I didn’t need, and was able to readjust my plans accordingly. And this was an excellent opportunity to learn! So not such a bad thing, really.

Questions for myself

  • How far to product do I think I am?
    • My opinion on this has shifted. I think now that I might be able to descope my original plan, and potentially get to product earlier. Particularly now that I am spending more time at home and don’t have a daily 3 hour commute during the week, I have more time, energy and opportunity for this hobby. So:
      • by end April I should be mostly done with NAT punchthrough
      • by end May I should have finished NAT punchthrough and be making inroads on the godot_voxel procedural generation piece for an MVP
      • by end June I should have finished the procedural generation piece, and have added a bit of polish
    • At that point I can probably ship the first MVP!
  • But what about the second iteration?
    • Yes, the second iteration should have a few more bits and pieces. In particular, I’d like to implement:
      • multiple tokens per player (a token is a character object)
      • the ability of the peer-as-server to assign tokens
      • the ability of a player to ‘avatar in’ to a particular token
      • the ability of a player to ‘avatar out’ of a particular token, and merely zip around the map with 3rd person camera
    • This might take me to the end of this year to complete.
  • And the third? =)
    • I’d like probably for the ability to drop particular props into the game as well by the peer-as-server. And probably also a permissions system, where the peer-as-server can allow certain peer-as-client machines to also alter the world. You could probably expect me to be done with that towards the middle of 2021 or later.

In terms of other questions …

  • Have your learning priorities shifted over the last couple of months?
    • I will continue following the Godot lectures for GodotGetaway by Canopy Games.
    • I probably won’t take on other Godot courses at present.
    • I probably won’t do “learning months” staggered between “hack months” as was my plan back in January, but rather learn the minimum I need to continue to advance my project.

Further questions …

  • Do I need a break before jumping back into the project again?
    • Nope!

Actions

  • In terms of a two month time horizon:
    • April will be focused as much as possible on trying to get NAT punchthrough working.
    • May will be mop-up for NAT punchthrough, and I reserve the right to do something else here as well … I guess I’ll see how much runway I have here closer to the end of April!
  • I will try to adhere to Wednesdays and Sundays in terms of days to rest.
  • I will put my health and the safety of those around me first and foremost and reserve the right to step away from this project for a bit if I need to clear my head / help others out etc.