Grooming the backlog, planning, and starting January sprint

Today I groomed the backlog a bit for the sandbox project, closed the last sprint, and started a new one.

Previous Goals

The goals for the December sprint for the sandbox project were:

• Retrieve procgen data on join an instance
• Complete draft of payment service

I succeeded on both objectives – although I did reinterpret the second, in that I decided that I would integrate Stripe with the Organisation service instead.

New goals

The goals for the January sprint for the sandbox project will be:

• Add redis cache to Tokengraph
• Implement temporary join codes for instances
• Implement basic account limits

December retrospective

General morale

Generally optimistic. As I write this on the 9th of August 2022 things have been progressing well.

What went well?

This month I:

• Rendered persisted procedurally generated data successfully for a client joining an instance,
• Prototyped Payments by integrating Stripe with the Organisation service,
• Tidied up the client a bit + enabled instances to be persisted for an organisation even if no users are present therein,
• Added functionality to allow for exit from an instance in the client and to return to the join page in the instance management UI.
• Migrated the client to Godot 3.5.

What didn’t go so well?

Things went pretty well actually.

What’s the outlook?

Reasonable.

My basic plan is:

• Add Redis to Tokengraph. Since Tokengraph is intended to serve potentially thousands of requests per minute, I should really be leveraging Redis to keep track of individual jobs.
• Complete the motion on generating temporary join codes for instances.
• Implement basic limits + pay down some technical debt.

The road ahead

Per the roadmap, to reiterate the points from here:

Near term (from now through to the end of the January 2023 sprint).

• Integrate payments with the UI. Although I’ve prototyped payments, I still need to integrate this with the UI. This should be moderately straightforward to do now but it is not an objective for January. (February)
• Implement shareable instance links functionality. (January)
• Implement basic limits. (January)

Not so near term (from the February 2023 sprint onwards through to December 2023, i.e. likely Q4 ’22 -> Q2 ’23 in real time).

• Procgen CRUD actions. Can move / rotate / delete procedurally generated object groups. (March)
• Deployment of services. Deployment of all the things, leveraging terraform and other techniques. (March -> May)
• Improve development environment. Starting to become a hassle, I’ll look towards containerising a few things and improve upon my bash scripts for local development. (February -> April)
• Create several basic generators. Sketch several tpgns for basic testing (forest, city, road) and add these to the palette. (March)
• UI / UX polish. I did a very small amount of this in the current sprint, adopting an approach where I treated different parts of the UI as different “pages”, but I’ll need to do a fair bit more work in the client before I have something that looks vaguely okay, and not the scrappy mess it currently is. (March -> August)
• Instance configurability. Try to improve the user experience in an instance, eg by making the size of it configurable (and have this tied to limits for an organisation gated by subscription level). See if a terrain mesh can be set for an instance too maybe? Probably not too complicated but having a degree of basic configurability before creating an instance would be good. (June -> July)
• Test coverage for the client in place, and client coverage at 20% overall. n.b. the tech to measure gdscript test coverage leveraging GUT doesn’t currently exist, but it should hopefully by the time I get around to looking into it again.
• Rudimentary test harnesses in place to focus on controllers within the instance service, the user service, the organisation service, and the payment service respectively. Test coverage hovering at about 10% for each of these services, and ideally pushing past 20%.
• Basic content available to users in palette (users should have a reasonably varied range of options to choose from in order to populate their instances). Ideally I’d like to allow for users to upload their own assets / procgen algorithms etc, but this is waaaay beyond the scope of a first iteration / prototype. For the prototype I’ll manually add, wire up and configure a basic selection of canned assets, and try to architect things with an eye for future generalisation and extensibility.

Deferred

• Parametrised procgen + improved procgen experience. Probably not a top priority for the initial prototype, I’ll punt this further down the road. Certainly though having configurable generation within the UI of things like cities is definitely something I’m very keen to implement. Under this milestone is introducing the abstraction of a ServiceNode type in tpgn graphs for calling out to separate services (like the City Generator Service based loosely perhaps on this). (Deferred, 2024).
• Recomputation of procgen object group. Can recompute a procedurally generated object group with different parameters (maybe out of scope, to be decided). (Deferred, 2024).
• Procgen placement previews. Mouse over preview of where things will be moved / rotated to. This should be possible by judicious use of collision masks (because I don’t want avatared tokens to be blocked by pending clipboard pastes, if I end up networking the preview view), as well as using ray tracing to detect a collision as to where the token or object group will go – the same way I place these things currently, just not “locked in”. (Deferred, 2024).

Tidied up join game page slightly

Today I tidied up the join game page slightly and prepared the ground for generating temporary join codes.

In particular, I:

• Removed the “error message” column from the join game table list and opted to emit errors for incorrect passwords to a new InfoBox label instead,
• Added a new button column to generate instance specific codes. The idea will be to send a message to comms and then to process the response, so as to populate the same InfoBox with the temporary join code.

This is essentially what the Join page screen now looks like:

Still very rough and ready, but getting there. I’ll probably want to remove the Password column at some point, or at least do something a bit better with it. Currently it is a dumb column if there is no password for the instance, and a text field entrybox when there is (if I recall correctly). This UX doesn’t seem optimal, I’ll try to improve upon this going forward.

A separate question is whether every member of an organisation should be able to Generate Join Codes for a given instance, I believe the answer should be Yes; those who are not members of the organisation or who haven’t even registered would be potential consumers of such codes, afterall.

It might even be possible to widen the sales funnel a fair bit by compiling the client to wasm and serving it online. This way people wouldn’t even have to download the client at all, they could just be given a URL to join an instance, and they would instantly join. Alternatively they could go to mycoolsystem.com and see the Client rendered there. Getting a bit sidetracked here, but the basic point is that with deep-linking per aforementioned “The Mirror” PR here (https://github.com/godotengine/godot/pull/63631) it should be possible to use a url parameter with the temporary join code for a completely anonymous user to have easy entry into this system; all they would need would be a url provided by someone with an account.

Concurrency limits would apply to organisations, of course!

Planning transient instance link codes

Today seems like a good day to think again about how to implement transient instance link codes for running instances. The usecase is for a user that creates an instance and wants to share it with their friends, who may have separate accounts and/or not have seats within their particular organisation, or who might not have an account at all and would like to play anonymously.

One general approach could be that the user clicks “generate code” in the client, the client would send a message to comms, comms would then ferry that to the instance service and the instance service would generate a unique code for that instance in a new table with a new controller. This would then be returned to comms and comms would return it to the client, where it would be displayed for copying by the user and shared via eg Zoom / GChat etc. Users would then enter this code in their clients, which would communicate via comms with the instance service. The instance service would verify the code, respond with “authorised” to comms, and then comms would provide them access to the instance.

An issue with this approach is that ideally these codes should be transient. i.e. just one day things. PostGres does not support TTL though. This suggests an alternative approach, wherein comms generates an ephemeral code and stores it in Redis with a TTL of 1 day. Then there is no need to reach out to the instance service at all, everything can be kept with comms.

I think I might go with the latter approach, as it involves fewer network hops, and is closer to the desired implementation I would prefer to follow. It also echoes the approach used in Talespire, which also offers ephemeral 1 day TTL on campaign access codes.

Implemented exit to main instance list page

Today I:

• Saved the EditorUI branch as scene in the client,
• Implemented exit from a running instance back to the main instance list. This means that one can jump in and out of instances from within the client without having to exit and jump back in constantly, which is a good quality of life improvement.

One thing I noticed is that it appears that I have a transient bug where things are null when they shouldn’t be – this occurs when joining a persisted instance that was opened earlier by the same client. The issue in particular are client exceptions which are thrown which exits out of the running client, I think this is likely driven by Redis on Comms, i.e. state not being cleared properly or alternatively not set properly for certain edge cases.

Nonetheless good progress today.

Fixed issue where non-anonymous instances are deleted

Today I fixed an issue where instances for non-anonymous organisations, i.e. with organisationId !== -1 – were being deleted. If an organisation is non-trivial I want to persist instances if a user leaves the instance and there are no peers remaining therein.

It turns out that I just needed to use the sessionData attribute loggedInOrganisationId and set the peerMetadata organisationId attribute accordingly before creating the instance. That fixed the issue.

Merry Christmas

⁶ For to us a child is born,
    to us a son is given,
    and the government will be on his shoulders.
And he will be called
    Wonderful Counselor, Mighty God,
    Everlasting Father, Prince of Peace.
⁷ Of the greatness of his government and peace
    there will be no end.
He will reign on David’s throne
    and over his kingdom,
establishing and upholding it
    with justice and righteousness
    from that time on and forever.
The zeal of the Lord Almighty
    will accomplish this.

Isaiah 9:6-7 NIV

Merry Christmas!

The twelfth day of Christmas

On the twelfth day of Christmas, my true love sent to me

twelve pollen baubles

eleven foggy cows

ten towers a talling

nine petals a swaying

eight ducks a bobbing

seven posts a concealing

six flowers a flaming

five pelicans

four wings deftly waving

three emus chirping

two rhinos chatting

and a moonbeam falling atwixt a gum tree

The eleventh day of Christmas

On the eleventh day of Christmas, my true love sent to me

eleven foggy cows

ten towers a talling

nine petals a swaying

eight ducks a bobbing

seven posts a concealing

six flowers flaming

five pelicans

four wings deftly waving

three emus chirping

two rhinos chatting

and a moonbeam falling atwixt a gum tree

The tenth day of Christmas

On the tenth day of Christmas, my true love sent to me

ten towers a talling

nine petals a swaying

eight ducks a bobbing

seven posts a concealing

six flowers flaming

five pelicans

four wings deftly waving

three emus chirping

two rhinos chatting

and a moonbeam falling atwixt a gum tree