HowTo

From wiki
Revision as of 07:27, 12 August 2024 by Sysop (talk | contribs) (→‎pyMetaModel)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

This page in other languages: de

HowTo en

How to Properly Semantify Your Data Gathering and Querying Needs using a Semantified Wiki to create a fitting Knowledge Graph

This guide outlines a systematic approach starting with your Needs and ending up with a fully operational Semantic MediaWiki that you may use as a prototype or even for production.

Steps

Start with gathering your needs

You can do this systematically (smartRQM), semi-systematically or fully freestyle in natural languages. Just make sure you get to an agreement between the stakeholders about what is relevant and capture the needs in computer readable form.

Set Up Your Wiki

Utilize ProfiWiki and py-mediawikidocker to set up your Semantic MediaWiki environment. This provides a solid foundation for your semantified wiki in a few minutes.

Create a Context for your project

You'll find examples context right in this wiki at List of Contexts Currently there are 16 example Contexts available:

Contextsinceupdatedmaster
BookContext7 August 20248 August 2024https://contexts.bitplan.com/index.php/BookContext
CityContext22 August 201510 August 2024https://contexts.bitplan.com/index.php/CityContext
ConfIDentMetadataSchema15 April 20207 August 2024https://contexts.bitplan.com/index.php/ConfIDentMetadataSchema
CrSchema22 July 202021 March 2023http://cr.bitplan.com
FamilyContext13 January 20157 August 2024https://contexts.bitplan.com/index.php/FamilyContext
Infrastructure27 July 202421 August 2024https://contexts.bitplan.com/index.php/Infrastructure
MetaModel23 January 20157 August 2024https://contexts.bitplan.com/index.php/MetaModel
OpenSourceProjectsContext31 October 201718 December 2022http://contexts.bitplan.com/index.php/OpenSourceProjectsContext
Presentation10 January 201522 December 2017https://contexts.bitplan.com/index.php/Presentation
QueryContext19 March 202319 March 2023https://contexts.bitplan.com/index.php/QueryContext
ResearchContext4 August 20207 August 2024https://contexts.bitplan.com/index.php?title=ResearchContext
SMWCon30 October 20157 August 2024https://contexts.bitplan.com/index.php/SMWCon
TeachingSchema16 January 20235 April 2023https://contexts.bitplan.com/index.php/TeachingSchema
WebContext31 October 2017http://wiki.bitplan.com/index.php/WebContext
WikiContext19 August 20157 August 2024https://contexts.bitplan.com/index.php/WikiContext
SmartRQM1 January 19968 July 2024http://rq.bitplan.com

Generate Semantic Structure

Use the y-principle generator to automatically create:

  • Concept pages
  • Templates
  • Forms
  • Help pages
  • Properties
  • List pages

That the Topic/Properties and TopicLinks needed for your Context. This approach ensures consistency and reduces redundancy in your wiki structure.

You'll get visualizations of your structure in UML and a prototype Wiki content as a result

Convert Existing Content

If you have existing content (like the SMWCon List of Attendees), convert it to use the newly created semantic structure. This may involve:

  • Transforming existing content to semantic mediawiki pages (e.g by adapting existing templates or importing structured content)
  • Replacing free-text information with property assignments
  • Creating new pages for concepts that were previously only mentioned in free text
  • changing the Structure to faciliated linking between contents to create a true knowled graph

Maintain Consistency

Regularly review and update your semantic structure. As your domain evolves, make sure to:

  • Update the Context
  • Regenerate semantic elements and the diagrams as needed
  • Refactor existing content to match the updated structure

Tools

All tools being used are open source and have been used for many years in quite a few projects already.

wikipush Toolkit

wikipush toolkit at github

py-sidif

py-sidif is used to parse Simple Data Interchange Format Triple files SiDIF is a simplified Triple serialization format in which

dad loves mom

is a valid triple declaration and the technical details are a separate concern which is hidden. (note how the Resource Description Framework RDF does not follow this separation of concerns but hard codes the use the Hypertext Transfer Protocol into it's design)

pymediawikidocker

pymediawikidocker automates the docker based setups of (Semantic) MediaWikis and supports migration from version to version by running multiple compatible wikis in parallel as a cluster.

pyMetaModel

PyMetaModel is a python library that can handle Contexts that are defined based on the MetaModel either directly from Template:Sidif or ProfiWiki input

py-y-principle generator

The Python based Y-Principle generator supports separation of concerns along the Y-Principle: domain and technical aspects are combined via a matrix of generation tasks. This way we do not have to manually work on all technical aspects for all domains. The number of tasks is much lower if we only have to supply the rules for the generation. So if you e.g. have 10 domains and 10 technical elements instead of 10x10=100 tasks you only have to do 10 technical descriptions + 10 domain descriptions + 10 mapping rules - that is only 30 steps in total.

AI / ChatBots

With AI and Chatbots following this process is even simpler, faster and more effective.

Examples

BookContext

Gathering the Needs

In the WikiSon explanation

{{  Book  |  Lord of the Rings  |  0261102389  }}

is given as an example Wiki notation for referencing the book Lord of the Rings.

We have formally gathered the Requirements for this example as:

But could have also simply used some examples in natural language and get the help from an AI Chatbot.

Set Up Your Wiki

We are using this wiki as a demo the MetaModel is already available.

Context Creation

For the Context creation we asked Claude AI: Book_Experiment resulting in BookContext

CityContext

Gathering the Needs

City pages have been used to explain SemanticMediaWiki since 2012. To start we copy the Demo Pages locally:

wikipush -s smw -t contexts -q "[[Category:City]][[Demo:~*]][[Population::+]]" -wi -f
copying 13 pages from smw to contexts
1/13 (   8%): copying ... Demo:Amsterdam✅
...
copying image File:Tokyo-Tsukishima-0011.jpg ...✅
13/13 ( 100%): copying ... Demo:Würzburg✅

Set Up Your Wiki

We will use the existing https://smw.bitplan.com partial copy of https://www.semantic-mediawiki.org/.

copy needed base ProfWiki pages

First we need to make sure needed pages for ProfiWiki are available.

wikipush -s contexts -t smwcopy -q "[[Needed_for::ProfiWiki]]" -wi -f
copying 23 pages from contexts to smwcopy
1/23 (   4%): copying ... Template:!✅
2/23 (   9%): copying ... File:Actionicon.pngcopying image File:Actionicon.png ...✅
3/23 (  13%): copying ... File:Component.pngcopying image File:Component.png ...✅
4/23 (  17%): copying ... File:Element into input.pngcopying image File:Element into input.png ...✅
5/23 (  22%): copying ... File:Help Icon.pngcopying image File:Help Icon.png ...✅
6/23 (  26%): copying ... HowTo/de✅
7/23 (  30%): copying ... Template:IconLink✅
8/23 (  35%): copying ... Icons✅
9/23 (  39%): copying ... File:Index.pngcopying image File:Index.png ...✅
10/23 (  43%): copying ... Property:IsA✅
11/23 (  48%): copying ... Template:Link✅
12/23 (  52%): copying ... Property:Link target✅
13/23 (  57%): copying ... Main Page✅
copying image File:Actionicon.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Component.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Element into input.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Index.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Link.png ...✅
copying image File:Masterlogo.png ...✅
copying image File:Registry.png ...✅
14/23 (  61%): copying ... MetaModel✅
copying image File:Actionicon.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Component.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Element into input.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Index.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Link.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Registry.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
15/23 (  65%): copying ... Property:Needed for16/23 (  70%): copying ... Template:NeededFor✅
17/23 (  74%): copying ... Template:PageSequence✅
18/23 (  78%): copying ... ProfiWiki/NeededPages✅
copying image File:Actionicon.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Component.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Element into input.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Index.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
copying image File:Profiwikiicon.png ...✅
copying image File:Registry.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
19/23 (  83%): copying ... File:Profiwikiicon.pngcopying image File:Profiwikiicon.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
20/23 (  87%): copying ... File:Registry.pngcopying image File:Registry.png ...⚠️:file exists (to overwrite existing files enable the ignore parameter)
21/23 (  91%): copying ... Template:TemplateSequence✅
22/23 (  96%): copying ... Template:UtfIcon✅
23/23 ( 100%): copying ... Template:WhatLinksHere✅

generate the MetaModel pages

This can be done via command line or interactively.

Create Context

CityContext is based on the properties of the Demo pages but uses WikiSon notation.

Convert Content

The pages have properties in traditional notation e.g. Demo:Tokyo

{{City
|collection = 東京
}}
Tokyo (also known as [[Has title::東京]] or Tokio) is the capital city, the center of the Greater Tokyo Area and one of the [[Has prefecture::tokyo|47 prefectures]] of [[Located in::Japan]]. Its coordinates are [[Has coordinates::35.6833° N, 139.7667° E]]

The [[Population::8,967,665|population]] of its special wards accounts for over 8 million people, with the total population of the prefecture exceeding 13 million.

[[Category:City]]

SMWCon

SMWCon is the acronym for the Semantic Mediawiki conference The lessons learned from the SMWCon experience and best practices in Semantic MediaWiki (SMW) are the basis for this example

Historical Context: SMWCon and List of Attendees

The journey of semantification for SMWCon began with the creation of the first SMWCon page using SemanticMediaWiki properties to capture information about SMWCon Events. https://www.semantic-mediawiki.org/wiki/SMWCon_Fall_2011 is an example for such a SMWCon page.

The proper way to handle the catalog of properties (Metadata management) has been much debated.

A self-reflective, semantified version of the Attendees list, as proposed by Pierre Racine (see Metadata management) of the List of Attendees at SMWCon Fal 2015 was a step forward. Over the years, additions were made, but unfortunately, SMWCon didn't fully embrace the power of self -reflective semantification.

Gathering the needs

We need to build on the existing pages from the categories

Creating the Context

SMWCon is the Context description in computer readable format it is based on

and example properties such as:

As of 2024-08-03 we have simply asked Claude AI to create the wikipage with SiDIF format based on that input which only needed minor corrections and took a few minutes.

Lessons Learned from SMWCon

  1. Avoid Redundancy: Don't copy-paste content. Use semantic properties and queries to display information in multiple contexts.
  2. Eat Your Own Dog Food: If you're promoting semantification, use it extensively in your own wiki.
  3. Maintain Consistency: Regularly review and update your semantic structure to ensure it remains relevant and efficient.
  4. Automate Where Possible: Use tools like the y-principle generator to maintain consistency and reduce manual work.
  5. Embrace Self-Reflection: Implement a system where your wiki's structure is itself described semantically, allowing for easier management and evolution of your knowledge base.

Links


⚠️ LLM-generated content notice: Parts of this page may have been created or edited with the assistance of a large language model (LLM). The prompts that have been used might be on the page itself, the discussion page or in straight forward cases the prompt was just "Write a mediawiki page on X" with X being the page name. While the content has been reviewed it might still not be accurate or error-free.