Sounds good to me; I’ll have a crack at that tomorrow evening.
1 Like
Sounds good, will have another update shortly.
1 Like
Alright, I’ve hooked up the plugin with the existing components as far as I could see, those changes are ready for review here:
Let me know what needs fixing there, and what else needs doing for the Nest launch?
PS: Looks like I don’t have edit access to the scrum board, so I haven’t moved any cards or assigned myself there.
Great! @owen, can you review and merge into a new branch?
I’ve invited you to the org on GitHub, but you need to accept the invitation before it goes through.
Ah, I’d almost forgot, there’s a change to the discourse-webkit-components plugin that wants to be merged in as well, so that plugin on edgeryders.eu will want to be updated too.
Update here:
- Added new views to Topics component, which can now be reused as a featured Stories section or to display itemised list/cards
- Topics formatting fixed
- Users have a card view new
- New people’s components that parses and displays posts for bios w/o accounts
- Events & Topics components (as do topics) now fetch data by tag
Still to do (Monday/Tuesday):
- Merge with @gdpelican branch
- Hero component with background options
- Separate views from component logic
- Width options for images/text
- Make image/text blocks reusable
- Mobile formatting
1 Like
Thanks for the work so far!
I think we should move on to creating a component from the form. Would you like to take that on too? We can continue on an hourly basis.
Could you start by scoping that work, including some rudimentary design implementations to make it fit the current graphical profile?
1 Like
It’s updated by now 
2 Likes
I had the idea that, since YAML is very close to lists with “name: value” list items, we might also use hierarchical lists in Discourse to encode this data. But now I’m pretty sure that’s a bad idea … YAML looks pretty clean and list-like itself, and using YAML avoids having to write a custom parser for yet another format.
Anyway, what’s the status about this idea now? Means, how are structured data communicated from Discourse to the webkit?
The one thing that (in my view) is not acceptable on the longer term is the ad-hoc use of Discourse tags to represent structured data. This was done at some point for the dates of events (but for that, the calendar plugin we have here on Discourse is the preferable solution).
Undocumented ad-hoc tags that somehow influence software function are just a messy way to do things. Anyone cleaning up tags will certainly break things that way. At the minimum, these tags would have to be documented here in our Discourse Admin Manual. And, they would have to use a common prefix like webcontent-*. Even then, more than one or two such tags per topic will appear as messy in the Discourse user interface (though there are CSS ways to hide them).
tl;dr: If there are better ways than tags to encode structured information for use by the webkit, please use better ways. Otherwise I would have to use those better ways in the future 
in order to keep Discourse clean …
It’s a JSON, which looks very similar to the one for the form software. Switching to YAML later is probably pretty easy if we want to do that.
I’m not sure there is, at least I can’t think of a better way. Given this, we defiantly need to settle on a syntax for how we name tags for web content.
Here is the template example.
1 Like
Since that JSON file is a static file hosted alongside the webkit software, no issue. It’s out of the realm of Discourse. What I’m discussing here is the case where we want to manage structured data inside Discourse that is then finding its way on webkit sites.
For example, team lists for a project website, containing a name, title, bio, twitter handle etc. per team member. For that (1) editing has to be on Discourse as people want to change it without having to rely on developers – the whole point of the webkit, (2) editing has to be comfortable on Discourse and (3) hosting that content on Discourse must cause as little mess and disruption in Discourse itself as possible. In that case, tags are almost universally bad. YAML inside Discourse posts seems pretty good to me though (as an evolution of the experiment done here).
How is that handled currently?
Seems I have said it too nicely, again So what I mean, in direct terms, is this: Do not rely on tags. It creates a mess that I really don’t want here in Discourse. There is always a better way than tags (except exceptions …). If you can’t think of one, please ask me. Options include:
- Discourse plugins that manage custom fields for topics or profiles (similar to the Calendar plugin or the PayCoupons field)
- YAML blocks inside Discourse posts
- externally referencing individual posts (as done for the various translations of the multi-lingual NGI Forward façade website)
- a hidden custom field for topics that defines the role of that topic in webkit sites and effectively works like a webkit-specific tagging system, but is never shown in the tag directory, tag auto-suggest lists etc.
- …
As a rule of thumb, if a proposed tag is of no use to the users on the Discourse platform, it’s probably not a use case for a tag. And if a proposed tag has no chance to achieve ≥100 uses in its lifetime, it’s not a use case for a tag. The disadvantages of namespace cluttering are higher then.
In the case of events and speaker profiles, this is indeed currently handled with tags and by making one topic item (speaker, event, etc). We will have to keep doing it like that for the time being, for the NGI Summit site we’re too pressed for time to implement something else.
This is coming pretty late in the development phase tbh. I think it’s been pretty clear from the start that we intended to use tags for this purpose and I didn’t see any problem with it myself. We’ve already built the the plugin to cache using tags and the entire architecture is set up with this in mind. Do you have a solution in mind that we could implement without having to re-do what has already been done?
I’m not saying that we can’t change this eventually, but in the first iteration of the webkit, it’s going to be tags unless we can find a way to implement another solution without breaking budget and still delivering on time for both the summit and for Nadias outreach sites.
Emh, no. I think it’s pretty clear from the above that I strongly proposed YAML (or JSON) in Discourse posts for all flow of structured data from Discourse to webkit software. And then there was no reaction, so I just remembered today to bring this up again.
Let’s ascribe it to miscommunication. For the future: when I really want something adapted in a software design where it’s not my role to finally decide it, I usually frame it as a proposal and indicate that I really would like to see that happen. That’s just about being nice, playing part of the team rather than commanding people around or showing disregard for their software design skills. But I only propose something that way if I think it’s significantly better than what I have seen so far (and here that includes the event and team tagging in Owen’s earlier iteration of webkit-like software).
So please, for the future, take this kind of change proposal seriously. Otherwise there will be a mess later and we’ll have to change things around to tidy up, anyway. As for myself, I’ll try to remember that I better tell that something is unacceptable here in Discourse (rather than making a constructive proposal for an alternative that is then ignored as a mere suggestion). My role here is to tidy up and keep Discourse organized, while yours (for this project) is to find new uses for Discourse. Both has its place …
Tags are just a selection mechanism for topics. So by switching to another such mechanism, no need to re-do much of the implementation.
For example for events that are about displaying a feed of events collected on the Discourse platform anyway, the mechanism could be:
-
all topics from a category, such as “Internet of Humans → Events”, that have a date field set using the Calendar plugin
-
all topics tagged event plus project-ngi-forward
Both of these are mechanisms already in established use on the platform. One of them uses tags, but these tags already exist and are useful for users of the Discourse platform. So if one of these is a sufficient solution for the use case, no need to invent anything new or to introduce redundancy. “Find the system, follow the system.” In fact, the more a website is generated from organic content found in Discourse rather than from content specifically written for that website, the better – because that’s the unique ability of the webkit, beyond just serving as a text editor for website content.
However, for the events that seem to be the common case for the NGI Summit website (like this one), my above proposal of using YAML inside Discourse posts seems more appropriate. It’s exactly made for cases like this. I would put the whole conference schedule into one YAML block, similar to putting a whole team into one JSON block here. Reasons:
-
These are rather short session descriptions, not something that you expect Discourse users to extensively comment on. If you’d want comments inside Discourse for this content, of course topics would be the way to go.
-
They contain some structured information like “Location: […]” and “Speakers: […]” that lends itself to YAML much more than to a messy parsing of Markdown / HTML to extract that information from post content.
-
Since these are small chunks of content and usually will get no comments on Discourse, it is wasteful to represent them as topics in Discourse. Topics are the second-highest level of structure in Discourse (below categories), so creating a lot of topics where there is no need means a lot of structural clutter. It’s like having a PhD thesis with only first-level headings and paragraphs and nothing else. Instead, when structuring something, I always try to find the adequate means within the toolkit of whatever is available to structure information in a software.
I think that this point applies even when it’s about content that is “only” meant for being displayed on an external website, with little relevance inside Discourse. Because: this content has to be in a public category to be accessible by the webkit, so you can’t keep these topics out of people’s “New” and “Latest” lists. When using (for example) posts inside a single topic for this content, these lists would be much less cluttered for users. In order to make the webkit a useful Discourse-based CMS product for wider use eventually, I think that’s an important point to raise about its interaction with Discourse.
tl;dr Selection by tag is fine for tags that serve a purpose for Discourse users – so keep that feature around. But I’d generalize that topic selection mechanism to also allow filtering by category, by availability of a certain custom field, perhaps by topic author, perhaps by is-wiki state. The advanced Discourse search function is your friend for this. For all remaining cases, custom Discourse tags can probably be avoided by the YAML-inside-Discourse scheme I proposed (plus hardcoded topic-and-post references in the webkit site’s configuration file). No issue if you have to do the NGI Summit with the custom tagging as started now, but please try to make the transition directly after that.
P.S.: The use of the ngi-summit-stories tag is indeed a pretty good use case for a Discourse tag. Because it’s working on organic content of the platform that may appear in any category, and there are few tools beyond tags that could be used to mark it. My only comment here would be to not tie the tag name to a specific event, but to frame it with future re-uses of that tag in mind, including for users of the Discourse platform itself. So maybe #ioh-featured or #ioh-featured-story would do?
To separate matters a bit; let’s first address the collaboration misunderstanding for the future:
I read your post very differently. It was phrased as “been toying around”, and as a suggestion to Owen, not a hard requirement. Had I known that you were adamant about it, it would have been a very different story. I didn’t even read it as a proposal, it sounded more like a possible idea. I also did not understand from this post that it was instead of using tags, just that you proposed this as one possible way of getting structured data.
In this case, our misunderstanding stems from that you see it as much more of a problem than I do to clutter the tag namespace. Since the namespace already contains a myriad of seldomly-used tags and appears very ad-hoc, it seemed to me that this implementation would not mess with any carefully constructed order. Had it actually been the case that we already had a cleaner namespace, I might very well have made a different assessment.
For reference, I read this strong encouragement as being about using YAML rather than JSON in the config file displayed with the webkit. Indeed, I did not understand why this was a strong encouragement, as it seemed like a minor point. Oh well, now we know.
I will, but I have to understand a few things to be able to assess the advice, namely:
- How strongly you feel about it and why
- What you see the consequences would be of making the wrong choice
All of this said, I very much respect your drive to keep the platform structured. I just want to note that in this case, there was no way for me to know that you had this opinion about tags since it was not explicitly stated, nor implemented in practice.
Now, turning from the meta-discussion towards making this work…
Here I need more information. It is true but is it also true that other selection mechanisms are as easily exposed with an endpoint as tags? On this, I would like the advice of @gdpelican.
This would be good, except for that we want a single webkit site to be able to display content from many categories.
This could be a way of doing it, yes. But it is a bit harder for the end-user applying tags to end up on the webkit. It means the instruction to the end-user becomes a little bit more complicated. This also requires the webkit cache plugin to support querying a set of tags, and it requires rewriting the webkit react modules to passing an array of tags. These are not hard problems, so we could do it in some cases.
However, it becomes a bit cumbersome to always have to think about content curation from the perspective of combining Discourse tags. For example, a specific curation of articles for a slider on the webkit for a specific purpose – like showing an interesting cross-section of different Edgeryders projects. In this case, we need to be able to somehow mark these topics and the most out-of-the-box way to do that is with a tag. Of course, we could also have a hidden custom field available for all topics for moderators that would basically be like a separate set of hidden tags. Again, I would need to understand if this changes anything for how we can query for these topics, and how much work this would be to implement.
Here we have tricky impasse; some sessions would be useful to allow commenting on, to start the conversation before the session. Others are not. And if we don’t use one topic per session, we also can’t use the events plugin, which means that we lose access to good features like enforcing the date formatting we need to display on the front end.
Hmm, but you can suppress a category for showing up in “Latest”, right? I have done so for this category, which is being used to feed content to the NGI Summit website.
Understood, we will continue as is, and also continue this conversation to find a sane way to do this that satisfies all parameters.
Well, yes and no… I think it might be the case that the curation for the summit will be quite different in nature than the curation we as Edgeryders care about, for example. In the Summit case, Nesta is doing the curation with us.
That’s how I roll. I make an earnest suggestion, and if it does not work I’ll step up the game. No need to tell people what to do as long as I can persuade them. In this case I basically forgot to follow up on the issue for too long 
It is not yet in order. 
Some tags are properly chosen and documented (see), otherwise it’s just chaos imported from Drupal, with chaos added on top. When I find a bit of mindspace, budget and time for it, @anu and me will certainly clean it up.
(I deleted hundreds of tags already, but it doesn’t show …)
Ok, hmm. Never thought of that way to misunderstand it, but indeed it’s possible. Communication is hard. Nevermind …
Yes. If you can make a SQL / ActiveRecord database query to select topics you want from the database, then you can make an API endpoint to expose them.
tangential rant: API redundancy
This reminds me of the other discussion I got involved in regarding the webkit: that adding any redundant API endpoints, like for topic selection by tag, is not a good idea in my view. Because redundancy is never a good idea.
I don’t care strongly about this part, as it doesn’t mess up what Discourse users see. So do how you like … but maybe think about it again. Because I’m pretty sure that kind of software architecture will come to bite you later. It’s implausible to me why you would want to add something redundant to an API – a “nicer” interface for a special-purpose application or caching before there has been an established practical need for caching are not compelling reasons to me.
Again, I’d not see the webkit as a kind of fancy editor for text that should appear on websites. Sometimes it will have that role, but its USP is building websites from organic Discourse forum content. That’s its true power: enabling content-rich, dynamic websites with very little manual creation and maintenance effort. And for that, mechanisms like multi-tag topic selection will be needed at some time. But not necessarily now.
The most out-of-the-box way for this use case is a reference to a topic ID in the webkit site’s configuration file. Tags or other mechanism on the Discourse side are only needed when Discourse users need the ability to add or remove content. For content where the selection of topics never changes in the foreseeable furture, references in the webkit config file are the easiest way to go about it.
In the YAML/JSON-in-Discourse-posts method, you can enforce it using a JSON schema (also mentioned above somewhere). There is probably a YAML equivalent of that.
Another argument against the one-topic-per-session and one-topic-per-bio approach is the user experience of the editing process inside Discourse. Right now there are for example two tags webcontent-ngi-summit-2020-bio and webcontent-ngi-summit-2020-bio-small. Probably they have redundant content, such as the speaker’s picture. That’s an argument for reading the full set of data from one place and displaying pieces of that as required.
Just try editing biographies etc. for two hours here in Discourse, and think if you like how that works, or why not. When doing the NGI Forward website, the first to fetch content from Discourse, I noticed that the user experience on Discourse is important. Basically, how easy it is to find and navigate the web content, and how easy it is to do the most frequent modifications.
Indeed. Didn’t know that option, it might be a recent addition. Note, it’s not set for the category you mentioned.
Even if; that set of curated articles will still be a good basis to build on after the summit. Tag naming is a signal, foremost, to the user of the Discourse platform. (Because that’s what the feature is designed for, and hacking it is fine as long as it interferes with that original use not in significant ways.) If you can name the same tag differently so that it becomes more useful to the forum user, there’s little reason against it.
There are always multiple ways to do it right. Hope you get the general idea of why I’m arguing against interventions on the Discourse side that are geared towards pushing content to one single, special-purpose website. As much as possible, it’s better to make the website pull content from Discourse that is either totally organic, or made for the website but marked in ways established in Discourse, or known to the software via its configuration file by topic and post ID.
Yes, and that would indeed be a use-case. For example; I’m planning to use the webkit to build a new website for Blivande. One section would be “News from the community”, which would be a slider of recent and relevant topics. If topics are no longer relevant or recent, the tag should be removed so that they stop displaying on the website.
This is a good idea because better yet, I can ask Markus from Nesta to review the experience. I will make sure to do so.
Right, doh. This was me reading the options wrong, it is now.
Indeed, it is noted. Thanks!
A Discourse tag news is useful for Discourse users and the website, so yes, that’s a good example for using a tag. It can be utilized on the Discourse side as well, by displaying the tag page with a special template similar to how our blog on Discourse worked (currently broken due to an update). I just would not remove the tag afterwards, because it eliminates useful information and work. Rather make the website slider display the most recent n items tagged news.
A counterexample would be a tag like #website-newsslider that is always only applied to (say) five news items. Or even worse, tags with fine-grained control over the position in the slider: #website-newsslider-pos1, #website-newsslider-pos2 etc… That’s rather useless and annoying for Discourse users, and does not exploit the possible synergy of news usage in both Discourse and the website. It’s also more maintenance intensive, harder to understand, and would prevent community members from participating in the tagging effort.
Deciding between good and bad design for selection mechanisms is a bit like in LaTeX: when the software itself is intelligent enough to select, order, layout and place things, it’s a waste of time to try exerting manual control over that process. A laid-back attitude of letting (good) software do the magic while being happy with the results is more useful here. Similarly, the webkit and CMS tools in general are tools for time-efficient, rule-based organization of content. These tools should not provide fine grained manual control over individual bits and pieces. Where you need that, use a completely hand-coded website (or hand-coded website element, as will probably be possible with the webkit).
You get the idea …
This only takes care of the “recent” criteria, but there is also the “relevant” criteria. For example, let’s take the case of Blivande. We have makers and artists working on projects in the space that we might want to display on the website, and doing this through the webkit is a great use case as we can then draw interested people straight into a conversation.
However, some projects are very long lasting while others fizzle out and die before getting started. In this case, we would want to manually curate and keep the projects tagged that are still active or noteworthy, while in in-tagging others, even if they are more recent.
Yes, I agree, that’s indeed going too far.
Here I don’t agree. One reason for making the web kit is indeed to give the power of some degree of curation control of the website to non-developers. But it is important to strike a good balance.