Brainstorming about XWiki Discussions

[mflorea]

@surli

Definition of a discussion

Sounds good.

@surli

Need of a messaging application

Sounds good.

@surli

Need of Notifications quick actions

Sounds good (using an extension point)

@surli

Data representation of Discussions

I’m in favor of using the XWiki store (pages and xobjects). Extracting a reusable application-forum-store module from the Forum Application sounds like a good plan for me.

@surli

So AFAIU commenting a page is a discussion start for you? It’s inline with the previous remark on comments, now it’s not something we considered when writing this. What’s interesting here is that it would raise some ambiguities about the event starting point of the discussion.

We could consider that page comments are bound to the page create event, i.e. that they are not bound to a specific version of the page, but to the fact that the page exists (was created). So creating a page “automatically” creates a discussion thread in comments about that page.

Thanks,
Marius

[vmassol]

@mflorea

I’m in favor of using the XWiki store (pages and xobjects). Extracting a reusable application-forum-store module from the Forum Application sounds like a good plan for me.

And that could be located in another extension or in platform so that installing support for discussions doesn’t automatically install the Forum.

[vmassol]

@mflorea

We could consider that page comments are bound to the page create event, i.e. that they are not bound to a specific version of the page, but to the fact that the page exists (was created). So creating a page “automatically” creates a discussion thread in comments about that page.

Even better, we should probably create the “discussion thread forum” only when the first comment is done (when the first comment for the page needs to be saved) so that we don’t waste space with empty wiki pages.

[vmassol]

Here’s a counterpoint to the current direction of this thread (I’m playing the devil’s advocate to make sure we don’t discard too quickly this solution which could actually be the best).

I think it was nice to be able to have a discussion store located in any existing page, as xobjects. Right now comments are xobjects of the page to which they’re comments to and that’s nice. It’s nice because they can be considered metadata of the page and it’s nice that they are grouped together. It has interesting properties: for example you just need to delete the page or move/rename the page and the discussion/comments follow. No need to sync! No need to implement transactions that span different pages to make sure it’s atomic (to avoid corrupted data).

I think I like this better than having a new store for comments/discussions. Make it much harder to sync with the context (in the example above sync between the page and the comments for it). Of course there are concepts that don’t have pages right now, such as notifications. This can be fixed either by introducing pages for them (too costly probably) or by creating only pages for notifications that have discussions and storing the discussion in the page.

Then we can imagine all the discussion UIs we wish and that would take their data from the xobjects in pages. A forum would be a space and a forum thread would be a page. I think that having single pages containing only a single metadata has always been not optimal from a design POV (we did that just for scalabilty reasons but when you start with xwiki you want to have lots of objects in a single page and then you later learn that it doesn’t scale and you change your design).

So for me this is ideal. So how do we achieve this?

By fixing the scalability issue we have with xobjects (that we need to fix anyway for an infinity of other use cases) and refactor our store code to stop loading all xobjects when the document is loaded. Instead, provide APIs to load xobjects on demand (starting offset and count), modify the xobject editor to use a scalable paginated solution (LiveData?). Searching is not an issue and it already works fine.

I have the feeling this would push towards a better direction, it wouldn’t cost more and it would solve a lot more use cases. It would certainly simplify a lot of extensions we started developing and that require separate pages for holding metadata because of the xobject scalability issues.

WDYT?

[surli]

@vmassol

I think it was nice to be able to have a discussion store located in any existing page, as xobjects. Right now comments are xobjects of the page to which they’re comments to and that’s nice. It’s nice because they can be considered metadata of the page and it’s nice that they are grouped together. It has interesting properties: for example you just need to delete the page or move/rename the page and the discussion/comments follow. No need to sync! No need to implement transactions that span different pages to make sure it’s atomic (to avoid corrupted data).

+1

@vmassol

or by creating only pages for notifications that have discussions and storing the discussion in the page.

+1

@vmassol

A forum would be a space and a forum thread would be a page.

+1

@vmassol

By fixing the scalability issue we have with xobjects (that we need to fix anyway for an infinity of other use cases) and refactor our store code to stop loading all xobjects when the document is loaded.

I agree that’s something that would probably help us a lot for future improvments. Maybe that’s something we could push for next cycle as part of the stabilization process, and then work on discussions by considering we won’t have a scalability issue with xobjects.

[vmassol]

FTR the JIRA issue is https://jira.xwiki.org/browse/XWIKI-2874

[mleduc]

From the discussion above, I think we agree on two important points:

• we do not want to directly reuse the forum application as a backend for the discussion
• we agree to use xobjects to store the discussion entities (topics, answers…)

Starting from that, I propose the following plan (details below).

First, introducing a 'xwiki-platform-discussion` module containing:

• xwiki-platform-discussion-store-api: providing an API to interact with a discussion store
• xwiki-platform-discussion-store-default: providing a default implementation for the discussion storage
• xwiki-platform-discussion-ui providing
  • reusable pieces of UI to display discussions contextually
  • a main page to consult the discussions of the current user

Then, we can this new discussion module:

1. For the ActivityPub discussions
2. As a backend for the forum application
3. As a backend for the comments and annotations
4. On new interesting places such as in reactions to notifications, or for contextual discussions for a given version of a page

Functional requirements

It should be possible to interact with the discussions of a wiki, from the wiki itself, but also externally (for instance through the ActivityPub protocol).

We need to keep in mind the goal to be able to reuse the discussion module as a backend for various aspects of the wiki so its design must be as generic as possible.

A discussion is represented by a topic containing a list of answers.
The topic can be attached to an entity (a page, a page version, an event…) and has a description. This description can be user-generated content, or be automatically derived from the related entity (for instance the message of a notification), that way even if the attached entity is later removed the context of the discussion is still available.

Discussion Store

The main factor to choose how we store the discussion is whether we can address the problem of xobjects scaling in 13.X (https://jira.xwiki.org/browse/XWIKI-2874).

If we agree to work on xobjects scaling, then a discussion topic and its answers would be:

• if the related entity is a page: directly stored in the page
• if the related entity is not a page: in a hidden page lazily created at the start of the discussion and dedicated to hold a reference to the entity and to store its answers.

If we do not want to make xobjects scale, the principle stays the same, but we will need to created hidden pages for the discussion’s answer too.

Discussion UI

If we want to quickly arrive at a usable result, I suggest keeping a design close to the one of the forum application.
That way, we can hope to be able to reuse design choices made for the forum, and it would also make the reuse of the discussion application as a backend for the forum easier.

It should contain macros to:

• display a preview of a discussion
• display a paginated discussion
• create a discussion
• add an answer to a discussion
• display a global list of all the discussion related to a user

If we agree on this general direction, I’ll update the discussion design draft with a more formal description of what I’ve said above.

WDYT?

[surli]

@mleduc

If we do not want to make xobjects scale, the principle stays the same, but we will need to created hidden pages for the discussion’s answer too.

I suggest that we start with that approach, and to improve it maybe with a new storage component whenever XWIKI-2874 will be fixed.

Else +1 for the rest of your post.

[vmassol]

@mleduc

First, introducing a 'xwiki-platform-discussion` module containing:

You may even want a xwiki-platform-discussion-api too. Example of APIs: add/remove reply to a discussion, mark discussion as read, add new discussion, delete discussion, move discussion, close discussion, etc.

@mleduc

It should be possible to interact with the discussions of a wiki, from the wiki itself, but also externally (for instance through the ActivityPub protocol).

So you could also need a REST API module to provide higher level REST API than the ones that operate on pages (if you change the store they won’t work anymore).

@mleduc

A discussion is represented by a topic containing a list of answers.

Yes we need to be clear about terminology. I don’t understand why you use 2 similar words: Discussion and Topic. If a discussion only has a topic then they are the same, no?

I think this is missing a level: the level of the forum which is a set of topics.

Also I wouldn’t use answers which has a connotation and is not generic enough. I would use a more neutral term like Replies.

So more like:

Discussion
  |_ Topic1
    |_ Reply1
    |_ ReplyN
  |_ TopicN

@mleduc

The topic can be attached to an entity (a page, a page version, an event…) and has a description

I think that’s the Discussion, not the topic.

@mleduc

If we do not want to make xobjects scale, the principle stays the same, but we will need to created hidden pages for the discussion’s answer too.

@surli

I suggest that we start with that approach, and to improve it maybe with a new storage component whenever XWIKI-2874 will be fixed.

I think it would be better to start to evaluate this work (not do it yet) to know how much it would cost to implement it. Then we can decide what we do.

@mleduc

If we agree on this general direction, I’ll update the discussion design draft with a more formal description of what I’ve said above.

Sounds good to me.

Note: we need to see the proposal of the APIs.

[mleduc]

@vmassol

You may even want a xwiki-platform-discussion-api too. Example of APIs: add/remove reply to a discussion, mark discussion as read, add new discussion, delete discussion, move discussion, close discussion, etc.

+1 (more on that below)

@vmassol

So you could also need a REST API module to provide higher level REST API than the ones that operate on pages (if you change the store they won’t work anymore).

+1

@vmassol

Yes we need to be clear about terminology. I don’t understand why you use 2 similar words: Discussion and Topic. If a discussion only has a topic then they are the same, no?

I think this is missing a level: the level of the forum which is a set of topics.

I propose the three concepts of Forum, Discussion and Reply:

• Forum: a group of discussions
  • id: String
  • name: String
  • administrators: List
  • creation date: Date
  • archived: boolean
• Discussion: a group of replies
  • id: String
  • topic: String
  • Entity id (optional): Entity
  • Forum id (optional): Forum
  • administrators: List
  • creation date: Date
  • archived: boolean
• Reply: a message on a discussion
  • id: String
  • message: String
  • Discussion id: String
  • author: UserReference
  • creation date: date
  • update date: date
  • archived: boolean

@vmassol

I think it would be better to start to evaluate this work (not do it yet) to know how much it would cost to implement it. Then we can decide what we do.

I’ll have a look but it’s probably safer if someone with more hindsight on the subject evaluates the work as well.

@vmassol

Note: we need to see the proposal of the APIs.

This is a tentative API, for space reason I avoided to specify the rights and error management (e.g., what if a forum containing discussions is removed).

Global design rules:

• All lists are paginated
• Operations that might fail to get a results returns Optional

// Create
Forum createForum(String name)
Discussion createDiscussion(String topic)
Discussion createDiscussion(String topic, Entity entity)
Discussion createDiscussion(String topic, Forum forum)
Discussion createDiscussion(String topic, Entity entity, Forum forum)
Reply createReply(String message, Discussion topic)

// Read
List<Forum> getForums(int offset, int limit)
Optional<Forum> getForum(String forumId, int offset, int limit)
List<Discussion> getDiscussions(Forum forum, int offset, int limit)
List<Discussion> getDiscussions(Entity entity, int offset, int limit)
List<Discussion> getDiscussions(Forum forum, Entity entity, int offset, int limit)
Optional<Discussion> getDiscussion(String discussionId)
List<Reply> getReplies(Discussion discussion, int offset, int limit)
Optional<Reply> getRepliy(String replyId)

// Update
void updateForum(Forum forum);
void updateDiscussion(Discussion discussion)
void updateReply(Reply reply)
void archiveForum(Forum forum)
void archiveDiscussion(Discussion discussion)
void archiveReply(Reply discussion)
void moveDiscussion(Discussion discussion, Forum target)

// Delete
void deleteForum(Forum forum)
void deleteDiscussion(Discussion discussion)
void deleteReply(Reply reply)

[vmassol]

@mleduc

I propose the three concepts of Forum, Discussion and Reply:

Note that when I said I think this is missing a level: the level of the forum which is a set of topics., I didn’t mean to use the word Forum. It might not be generic enough. We need to a word to represent a group of discussions. For example, I don’t know how you plan to implement the notification discussions. I guess you’ll have one Discussion per notification and all the notification discussions will fall under a “Forum”? Is “Forum” the best word here? It could well be, I’m just asking.

Other stuff I noticed:

• a group of discussions so Discussion Group could be better maybe no? Might be more explicit?
• topic: String. Is it a discussion or a topic? We need only one word. Maybe you meant “title”?
• archived: too limited IMO since it can only be true or false. Maybe state which is more extensible?
• Is this list of properties the minimal list or supposed to be the full list. Should it be extensible and be extended by users of the discussion api? For ex, the Forum app might want to introduce the concept of moderators.
• What is the name of the forum? Don’t we need a summary too? Same question for Discussion.
• Entity id seems too restrictive to me. I think it’s a “source” information (String) that could be anything, right? It would need a format too. For example a notification doesn’t have an entity reference.
• a message on a discussion so maybe it should be named Message?
• “message” --> do you mean “content”?
• Why update date on a message and not on discussion or forum? what is it for?

Thus maybe we should have:

• Discussion Group > Discussion > Message

?

@mleduc

Reply createReply(String message, Discussion topic)

It doesn’t say to which message I’m replying to. Don’t forget the comment use case for example.

Also make sure we can implement the annotation use case, maybe as an extension.

@mleduc

void archiveReply(Reply discussion)

s/discussion/reply

Other ideas:

• How do you pin a forum topic? I think it should be in an extension used by the forum but you need to prove that the new API can be used by the forum application. Thus the data model needs to support extensibility since IMO we want something quite minimal.

[mleduc]

We had chat with @surli and we converged in the same direction.
What we propose is to substitute the forum for a more generic “discussion context” which seems to be very close to a “discussion group”.

A discussion context contains:

• an entity type: defining the type of referenced entity
• an entity reference: defining the serialized reference if the entity (for instance if type is “document” the reference would be document reference, and if the type is “event” the reference would be a reference id).
• a description
• a title

The main difference might be that a discussion can be part of several discussion contexts.
This many to many relationship between discussion contexts and discussions ease some use cases. For instance a discussion started from an update event on a page could be linked to two discussions contexts, one referencing the event, and a second one referencing the page.

Consequently the notion of entity in the discussions becomes deprecated and should be removed.

@vmassol

Is this list of properties the minimal list or supposed to be the full list. Should it be extensible and be extended by users of the discussion api? For ex, the Forum app might want to introduce the concept of moderators.

For the rights, I think it is easier if the discussion extension comes with a right system, but indeed it might be useful to also add a properties map to the discussion contexts/discussions/messages to store unanticipated fields for specific integrations of the discussions.

Regarding the rights, what I have in mind is to add the notions of:

• administrators: can perform so administration tasks such as removing messages, archiving a discussion…
• “view” actors: the list of users/groups that can view the discussion
• “write” actors: the list of users/groups that can add messages to the discussion

I’m not sure how what is the best vocabulary for these concepts.

This also needs to take into account to allow/disallow messages to be added from external sources such as the fediverse. I’m not sure of how to do that right now.

@vmassol

archived: too limited IMO since it can only be true or false. Maybe state which is more extensible?

Is the state value free, and defined by the application users? Or should we propose a larger set of predetermined values?

@vmassol

Why update date on a message and not on discussion or forum? what is it for?

The update and creation date should be in all the entities. This can be useful on the UI, to sort the discussions/messages, or to display the date of the last activity in a discussion for instance.

For the rest, I agree with your observations.

I’m in the process of updating https://design.xwiki.org/xwiki/bin/view/Proposal/Discussions to align with the direction of the discussion.
I’ll:

• update the vocabulary: DiscussionContext > Discussion > Message
• detail the notion of rights
• add an attribute in the messages to allow for nested messages
• add the notion of pined discussion (maybe by extension)
• update the APIs accordingly

WDYT ?