Updates & New Devlog

[Jubal]

So, as some of you may know, I'm now a PhD student at the University of Vienna, which rather changes the game for the CPP. I'm planning to leave the old testing version up on Exilian, but the newer versions of it are ultimately likely to be set up in a rather better protected environment by the university, and are using rather different technologies - Neo4Jr ather than SQL for the database-end, though still PHP at the front.

Nonetheless, I want to put a devlog here still to record what I'm doing with the project - I want to keep a public-facing record of where things are going, partly to keep myself on track and partly because I think having some chance for folk to publicly engage with my work is good from an academic transparency perspective. As such, here's the new devlog thread! I'm happy to post screencaps of what I'm doing or provide people with test-drive versions of the software, but I'm not going to be running a public version in the near future thanks to the cost & difficulty of setting up a server running neo4j etc.



Today's work:

• Removed testing features from the mini-biography reference system, ready to create the proper node property for biographies.
• Added extra notes to the notes page, especially on how the Identity node-type works

[Jubal]

Today:

• Added textboxes to the edit page so I can change people's name and edit their biography. I can now in theory kinda make referenced biographies! (I still need to add the feature to add new books)
• Added the biography attribute to the database
• Added the biography attribute to the notes page

Tomorrow's job should be adding the "event" node and some basic features for editing it, though I need to also get on and do the "add new person" feature too.

[Glaurung]

Good to hear this is under way. If there's a way for me to do user testing, I'd be happy to put some time into it.

[Jubal]

Thankyou! It'll be a while before it's ready for that, I think - my current plan is to add event nodes, finish the biography nodes, add the location nodes, and then input all the data from the English version of the Chronicle of Giorgia Lasha and his Time (selected because it's the shortest chronicle covering the period) - at which point I then have a tiny dataset I can manipulate and test out some more interesting functionality with, which I guess will be the stage where extra testing might help.

[Jubal]

Finally, after some hiccups, the bibliography "add new books" system is nearly sorted (that is, the system that creates links between people's pages and the books that are referenced in their mini-bibliography). I've been wrestling with the last major hiccup for the past hour, which turned out to be a feature of Neo4j not having implicit integer to string conversion whereas PHP does, which managed to confuse my code a little. I currently don't have a feature for removing books, but that's a fairly low concern at this point since I'm unlikely to need it often. The whole system will also be needed for events: I was considering trying to integrate events better into people's pages, but I think it may be simpler for events to stay pretty firmly as their own separate thing/node type with their own pages, and just have tags/links between people and the events at which they were present.

[Glaurung]

The whole system will also be needed for events: I was considering trying to integrate events better into people's pages, but I think it may be simpler for events to stay pretty firmly as their own separate thing/node type with their own pages, and just have tags/links between people and the events at which they were present.

My feeling is that events should almost certainly be a separate data entity (node?) from people: one event can have multiple people involved (or possibly none at all); each person is likely to participate in multiple events. In terms of UI, a person's page might have a header section with biographical details, and then a list of events, while an event page could include a list of the people present.

[Jubal]

Yes - they've always been going to be a separate node type, with a separate edit page (with links to people and location objects). The question was more how well their UI could link into the person pages, and how that interacted with the reference links, which are one of the more headache-inducing parts of the whole system; one can't link from a link, only from a node, which sometimes makes referencing "why I've put this link here" tricky, and also loading in all the references from an event alongside the ones for the person and then sorting them out into some sort of coherent system and re-ordering them to get a proper ref & bibliography section is a headache I've decided is best avoided.

[Glaurung]

Yone can't link from a link, only from a node, which sometimes makes referencing "why I've put this link here" tricky

Perhaps make the links into nodes of their own (obviously of a new "relationship" or "link" type) so that you can attach relevant data to them, and then connect these "link" nodes to the relevant "ordinary" nodes? I don't know what your database system allows or encourages; I'm used to traditional relational (a.k.a. SQL) databases. But if you're wanting to record, not just that person A was at event B, but that we know this because of reference C, then it seem to me that the "link" between A and B has acquired node-like characteristics.

[Jubal]

I could do that, I just feel like it might make the whole system less intuitive and more cumbersome to work with from my perspective. I can attach properties to links, so I suspect it might be easier (and perhaps more transparent for the user) if I add some of this information as prose notes in the properties. The only technical thing I can think of where the concept of link-nodes might help would be that it would make it easier to switch certain whole sources on and off as to their impact on the database - but in general I'm not sure that's a useful enough feature for the workload at this stage, even if it might be generally fun to include.

[Glaurung]

I could do that, I just feel like it might make the whole system less intuitive and more cumbersome to work with from my perspective. I can attach properties to links, so I suspect it might be easier (and perhaps more transparent for the user) if I add some of this information as prose notes in the properties.

Ah, OK. I assumed from what you said earlier that a link was purely "A relates to B" and couldn't hold any more information, but link properties sound useful.

[Jubal]

Yes, link properties can be set and used to look things up - which I guess is more how things might work in an SQL database in the first place. They'll probably need numerical keys to fit onto references somehow, which will be a headache to say the least, but probably a smaller one than having to have linking nodes for every link I want to make...

I think the next thing to do is press on and get event nodes set up, anyway. Once I've got some data to play with it might help.

[Jubal]

Just got the "add person" feature working finally - it now reliably finds the highest unused id from the database, and you can plug in some book links, a name, and a mini-biography. Identity links etc still need to be added afterwards, which is a bit laborious - I might add in a list with some some checkboxes for common identities to make it easier to add several linkages in one go.

[Glaurung]

it now reliably finds the highest unused id from the database

Highest used ID (then adds 1), or lowest unused ID, perhaps?

Pedantry aside, well done - it's obviously an important step forwards.

[Jubal]

Aaaand now some bad news, which is that the id system has been storing all the numbers as strings, and I need to rewrite a LOT of code - basically all my cypher queries in the system so far - to treat the IDs as numbers or they won't go into order properly in the database query (I could alternatively just sort them in the PHP, but I suspect making them all numbers makes more sense.)

[Glaurung]

Oh dear, that sounds painful - string / number conversions are very rarely fun, or good for system performance