Migrate From Vimwiki To Markdown Syntax


Vimwiki supports markdown and I switched to this because:

Fixing My Config

Vimwiki expects some options set before it loads (ref). With lazy, I use init like:

{ 'vimwiki/vimwiki',
  branch = 'dev',
  dependencies = {'mattn/calendar-vim'},
  init = function()
    vim.g.vimwiki_list = {
      {path = "~/vimwiki", path_html= "~/vimwiki/public_html", auto_tags = 1, auto_diary_index = 1, syntax = 'markdown', ext = '.md'},
      -- experimental folder for markdown conversion
      {path = "~/projects/vimwiki", path_html= "~/projects/vimwiki_markdown/public_html", syntax = 'markdown', ext = '.md'}, }
    vim.g.vimwiki_markdown_link_ext = 1
    vim.g.vimwiki_stripsym = ' '
    vim.g.vimwiki_global_ext = 0

Pandoc Conversion

I tested pandoc for conversion with the file:

== Title ==

- [o] list item i:
  - [.] sublist
    - [X] sub sub
    - [ ] sub usb
    - [ ] sub sub1
  - [X] final

TODO: task 1

%% a comment I've left to address later

I'm a normal paragraph.

[[link to file]]

  ls -lah
  find . -iname '*.wiki'

Another code block

  ls -lah
  find . -iname '*.wiki'

Running pandoc --from vimwiki --to commonmark_x trial.wiki gave me:

╰─$ pandoc --from vimwiki --to commonmark_x trial.wiki
## Title {#Title}

- []{.done2}list item i:
  - []{.done1}sublist
    - []{.done4}sub sub
    - []{.done0}sub usb
    - []{.done0}sub sub1
  - []{.done4}final

[TODO:]{.todo} task 1

I\'m a normal paragraph.

[link to file](link to file "wikilink")

``` bash
  ls -lah
  find . -iname '*.wiki'

Another code block

      ls -lah
      find . -iname '*.wiki'

I had these problems with the conversion:

I'd combine sed with pandoc to get what I wanted.


I replaced the comment identifier %% with TODO: comment to the prevent losing them during conversion. I also replaced code blocks without a language i.e. {{{ to have bash i.e. {{{bash and I'd replace this when editting later.

╰─$ echo -e "%% save me\nNormal line %% with ending\n{{{" \
      | sed -r -e 's/\{\{\{$/\{\{\{bash/g' \
               -e 's/%%/TODO: comment/g'
TODO: comment save me
Normal line TODO: comment with ending

Post processing

I changed the vimwikie-converted linked i.e. [link_to_file](link_to_file "wikilink") to use the file name i.e. [link_to_file](link_to_file.md).

echo -e "[link_to_file](link_to_file \"wikilink\")" \
  | sed -r -e 's/(\[.*\])\(([^#]*)((.*) "wikilink")\)/\1\(\2.md\4\)/g'

I converted all checklists to github's checklist syntax with:

echo -e "- []{.done4}done\n- []{.done0} not_done" \
  | sed -r -e "s/\[\]\{\.done[0-3]\}/\[ \] /g" \
           -e "s/\[\]\{\.done4\}/\[X\] /g"

and also unquoted all ' characters with:

echo -e "\\'" | sed -e "s/\\\'/\'/g"

Here's the final post processing step:

  sed -r -i -e 's/(\[.*\])\(([^#]*)((.*) "wikilink")\)/\1\(\2.md\4\)/g' \
    -e "s/\\\'/\'/g" \
    -e "s/\[\]\{\.done[0-3]\}/\[ \] /g" \
    -e "s/\[\]\{\.done4\}/\[X\] /g" \

Putting It All Together


set -euo pipefail

# files to change extension from to md format
readarray -d '' mv_files < <( \
  find . \( -iwholename '*diary/*.wiki' \
    -or -iwholename '*unorganized_things/*.wiki' \
    -or -iwholename '*design_docs/*.wiki' \
    -and -not -iname 'diary.wiki' \
    -and -not -iname 'index.wiki' \) \

for file in "${mv_files[@]}"; do
  mv "$file" "$md_file"

readarray -d '' files < <(find . -name "*.wiki" -print0)
for file in "${files[@]}"; do
  sed -r -e 's/\{\{\{$/\{\{\{bash/g' -e 's/%%/TODO: comment/g' "$file" | pandoc --from vimwiki --to commonmark_x -o "$md_file"
  sed -r -i -e 's/(\[.*\])\(([^#]*)((.*) "wikilink")\)/\1\(\2.md\4\)/g' \
    -e "s/\\\'/\'/g" \
    -e "s/\[\]\{\.done[0-3]\}/\[ \] /g" \
    -e "s/\[\]\{\.done4\}/\[X\] /g" \
  rm "$file"