HowTo

This page in other languages: de

HowTo en

How to Properly Semantify Your Data Gathering and Querying Needs using a Semantified Wiki to create a fitting Knowledge Graph

This guide outlines a systematic approach starting with your Needs and ending up with a fully operational Semantic MediaWiki that you may use as a prototype or even for production.

Steps

Start with gathering your needs

You can do this systematically (smartRQM), semi-systematically or fully freestyle in natural languages. Just make sure you get to an agreement between the stakeholders about what is relevant and capture the needs in computer readable form.

Set Up Your Wiki

Utilize ProfiWiki and py-mediawikidocker to set up your Semantic MediaWiki environment. This provides a solid foundation for your semantified wiki in a few minutes.

Create a Context for your project

You'll find examples context right in this wiki at List of Contexts Currently there are 16 example Contexts available:

Context	since	updated	master
BookContext	7 August 2024	8 August 2024	https://contexts.bitplan.com/index.php/BookContext
CityContext	22 August 2015	29 October 2024	https://contexts.bitplan.com/index.php/CityContext
ConfIDentMetadataSchema	15 April 2020	7 August 2024	https://contexts.bitplan.com/index.php/ConfIDentMetadataSchema
CrSchema	22 July 2020	21 March 2023	http://cr.bitplan.com
FamilyContext	13 January 2015	7 August 2024	https://contexts.bitplan.com/index.php/FamilyContext
Infrastructure	27 July 2024	21 August 2024	https://contexts.bitplan.com/index.php/Infrastructure
MetaModel	23 January 2015	7 August 2024	https://contexts.bitplan.com/index.php/MetaModel
OpenSourceProjectsContext	31 October 2017	18 December 2022	http://contexts.bitplan.com/index.php/OpenSourceProjectsContext
Presentation	10 January 2015	22 December 2017	https://contexts.bitplan.com/index.php/Presentation
QueryContext	19 March 2023	19 March 2023	https://contexts.bitplan.com/index.php/QueryContext
ResearchContext	4 August 2020	7 August 2024	https://contexts.bitplan.com/index.php?title=ResearchContext
SMWCon	30 October 2015	7 August 2024	https://contexts.bitplan.com/index.php/SMWCon
TeachingSchema	16 January 2023	5 April 2023	https://contexts.bitplan.com/index.php/TeachingSchema
WebContext	31 October 2017		http://wiki.bitplan.com/index.php/WebContext
WikiContext	19 August 2015	7 August 2024	https://contexts.bitplan.com/index.php/WikiContext
SmartRQM	1 January 1996	8 July 2024	http://rq.bitplan.com

Generate Semantic Structure

Use the y-principle generator to automatically create:

• Concept pages
• Templates
• Forms
• Help pages
• Properties
• List pages

That the Topic/Properties and TopicLinks needed for your Context. This approach ensures consistency and reduces redundancy in your wiki structure.

You'll get visualizations of your structure in UML and a prototype Wiki content as a result

Convert Existing Content

If you have existing content (like the SMWCon List of Attendees), convert it to use the newly created semantic structure. This may involve:

• Transforming existing content to semantic mediawiki pages (e.g by adapting existing templates or importing structured content)
• Replacing free-text information with property assignments
• Creating new pages for concepts that were previously only mentioned in free text
• changing the Structure to faciliated linking between contents to create a true knowled graph

Maintain Consistency

Regularly review and update your semantic structure. As your domain evolves, make sure to:

• Update the Context
• Regenerate semantic elements and the diagrams as needed
• Refactor existing content to match the updated structure

Tools

All tools being used are open source and have been used for many years in quite a few projects already.

wikipush Toolkit

wikipush toolkit at github

py-sidif

py-sidif is used to parse Simple Data Interchange Format Triple files SiDIF is a simplified Triple serialization format in which

dad loves mom

is a valid triple declaration and the technical details are a separate concern which is hidden. (note how the Resource Description Framework RDF does not follow this separation of concerns but hard codes the use the Hypertext Transfer Protocol into it's design)

pymediawikidocker

pymediawikidocker automates the docker based setups of (Semantic) MediaWikis and supports migration from version to version by running multiple compatible wikis in parallel as a cluster.

pyMetaModel

PyMetaModel is a python library that can handle Contexts that are defined based on the MetaModel either directly from Template:Sidif or ProfiWiki input

py-y-principle generator

The Python based Y-Principle generator supports separation of concerns along the Y-Principle: domain and technical aspects are combined via a matrix of generation tasks. This way we do not have to manually work on all technical aspects for all domains. The number of tasks is much lower if we only have to supply the rules for the generation. So if you e.g. have 10 domains and 10 technical elements instead of 10x10=100 tasks you only have to do 10 technical descriptions + 10 domain descriptions + 10 mapping rules - that is only 30 steps in total.

AI / ChatBots

With AI and Chatbots following this process is even simpler, faster and more effective.

Examples

BookContext

Gathering the Needs

In the WikiSon explanation

{{  Book  |  Lord of the Rings  |  0261102389  }}

is given as an example Wiki notation for referencing the book Lord of the Rings.

We have formally gathered the Requirements for this example as:

• Project: BookProject
• Requirements: BookWithISBN
• Acceptance Criterion: BookLordOfTheRings

But could have also simply used some examples in natural language and get the help from an AI Chatbot.

Set Up Your Wiki

We are using this wiki as a demo the MetaModel is already available.

Context Creation

For the Context creation we asked Claude AI: Book_Experiment resulting in BookContext

CityContext

Gathering the Needs

City pages have been used to explain SemanticMediaWiki since 2012. To start we copy the Demo Pages locally:

wikipush -s smw -t contexts -q "[[Category:City]][[Demo:~*]][[Population::+]]" -wi -f
copying 13 pages from smw to contexts
1/13 (   8%): copying ... Demo:Amsterdam✅
...
copying image File:Tokyo-Tsukishima-0011.jpg ...✅
13/13 ( 100%): copying ... Demo:Würzburg✅

Set Up Your Wiki

We will use the existing https://smw.bitplan.com partial copy of https://www.semantic-mediawiki.org/.

copy needed base ProfWiki pages

First we need to make sure needed pages for ProfiWiki are available.

wikipush -s contexts -t smwcopy -q "[[Needed_for::ProfiWiki]]" -wi -f
copying 23 pages from contexts to smwcopy
1/23 (   4%): copying ... Template:!✅
2/23 (   9%): copying ... File:Actionicon.pngcopying image File:Actionicon.png ...✅
3/23 (  13%): copying ... File:Component.pngcopying image File:Component.png ...✅
4/23 (  17%): copying ... File:Element into input.pngcopying image File:Element into input.png ...✅
5/23 (  22%): copying ... File:Help Icon.pngcopying image File:Help Icon.png ...✅
6/23 (  26%): copying ... HowTo/de✅
7/23 (  30%): copying ... Template:IconLink✅
8/23 (  35%): copying ... Icons✅
9/23 (  39%): copying ... File:Index.pngcopying image File:Index.png ...✅
10/23 (  43%): copying ... Property:IsA✅
11/23 (  48%): copying ... Template:Link✅
12/23 (  52%): copying ... Property:Link target✅
13/23 (  57%): copying ... Main Page✅
copying image File:Actionicon.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Component.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Element into input.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Index.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Link.png ...✅
copying image File:Masterlogo.png ...✅
copying image File:Registry.png ...✅
14/23 (  61%): copying ... MetaModel✅
copying image File:Actionicon.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Component.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Element into input.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Index.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Link.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Registry.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
15/23 (  65%): copying ... Property:Needed for✅
16/23 (  70%): copying ... Template:NeededFor✅
17/23 (  74%): copying ... Template:PageSequence✅
18/23 (  78%): copying ... ProfiWiki/NeededPages✅
copying image File:Actionicon.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Component.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Element into input.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Index.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Profiwikiicon.png ...✅
copying image File:Registry.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
19/23 (  83%): copying ... File:Profiwikiicon.pngcopying image File:Profiwikiicon.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
20/23 (  87%): copying ... File:Registry.pngcopying image File:Registry.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
21/23 (  91%): copying ... Template:TemplateSequence✅
22/23 (  96%): copying ... Template:UtfIcon✅
23/23 ( 100%): copying ... Template:WhatLinksHere✅

generate the MetaModel pages

This can be done via command line or interactively.

Create Context

CityContext is based on the properties of the Demo pages but uses WikiSon notation.

Convert Content

The pages have properties in traditional notation e.g. Demo:Tokyo

{{City
|collection = 東京
}}
Tokyo　(also known as [[Has title::東京]] or Tokio) is the capital city, the center of the Greater Tokyo Area and one of the [[Has prefecture::tokyo|47 prefectures]] of [[Located in::Japan]]. Its coordinates are [[Has coordinates::35.6833° N, 139.7667° E]]

The [[Population::8,967,665|population]] of its special wards accounts for over 8 million people, with the total population of the prefecture exceeding 13 million.

[[Category:City]]

SMWCon

SMWCon is the acronym for the Semantic Mediawiki conference The lessons learned from the SMWCon experience and best practices in Semantic MediaWiki (SMW) are the basis for this example

Historical Context: SMWCon and List of Attendees

The journey of semantification for SMWCon began with the creation of the first SMWCon page using SemanticMediaWiki properties to capture information about SMWCon Events. https://www.semantic-mediawiki.org/wiki/SMWCon_Fall_2011 is an example for such a SMWCon page.

The proper way to handle the catalog of properties (Metadata management) has been much debated.

A self-reflective, semantified version of the Attendees list, as proposed by Pierre Racine (see Metadata management) of the List of Attendees at SMWCon Fal 2015 was a step forward. Over the years, additions were made, but unfortunately, SMWCon didn't fully embrace the power of self -reflective semantification.

Gathering the needs

We need to build on the existing pages from the categories

• Event
• Talks

Creating the Context

SMWCon is the Context description in computer readable format it is based on

• https://www.semantic-mediawiki.org/wiki/List_of_Attendees
• https://wiki.bitplan.com/index.php/Attendee

and example properties such as:

• Talk status
• Attendee picture

As of 2024-08-03 we have simply asked Claude AI to create the wikipage with SiDIF format based on that input which only needed minor corrections and took a few minutes.

Lessons Learned from SMWCon

1. Avoid Redundancy: Don't copy-paste content. Use semantic properties and queries to display information in multiple contexts.
2. Eat Your Own Dog Food: If you're promoting semantification, use it extensively in your own wiki.
3. Maintain Consistency: Regularly review and update your semantic structure to ensure it remains relevant and efficient.
4. Automate Where Possible: Use tools like the y-principle generator to maintain consistency and reduce manual work.
5. Embrace Self-Reflection: Implement a system where your wiki's structure is itself described semantically, allowing for easier management and evolution of your knowledge base.

Links

• Metadata management in Semantic MediaWiki
• Working with properties in SMW

⚠️ LLM-generated content notice: Parts of this page may have been created or edited with the assistance of a large language model (LLM). The prompts that have been used might be on the page itself, the discussion page or in straight forward cases the prompt was just "Write a mediawiki page on X" with X being the page name. While the content has been reviewed it might still not be accurate or error-free.