How learning.fail Works

07/11/2024

Updated 08/07/2025

This place was created so I could do something I'm not comfortable with... write about stuff. Before I started writing, I needed a way to host my ramlbings. At first I wrote a big complicated web app in flask. Then I decided I didn't want to have to manage code a large code base and introduce security risks. Then I thought about using a cms like Wordpress but that also needs to be constantly patched or risk explotation. Finally I decided to keep it simple and stick to core technologies. Here are a few of my reasons for this:

Simple
- No complicated backend, databases or ui design elements.
- Easy to backup, should be be accomplished by simple cronjob.
- No dependcies on server configurations.
- An easy way to update and add content.
- No frameworks, core tech only.
Fast
- No analytics or other third party scripts.
- No ads.
- Low bandwidth.
- Host on a potato.
Secure
- No CMS, which requires maintance, patching, careful permissions.
- No Javascript required.
- No data handling (cgi gateways, databases, etc.) besides the http daemon.
  - I don't want to introduce new ways to break into my server.
  - I don't want to have to worry about bad code and 0days on a bunch of frameworks and dependencies.

What I did

I'm hosting on a small box that costs pennies to run. For content managment and rendering I chose to write all my documents in markdown files and generate the html from a script. This choice gives me a few advantages over just creating a new html file for each post.

I can create and change templates easily.
I don't have to worry about html and focus on the writing.
It is a common format and well documented standard which has libraries in most languages.
Provides just enough formating to keep the page clean and originized without the need for complicated frameworks and css styles.
It doesn't introduce complicated frameworks or potential vulnerabilities.

How it works

The generate.py script lives in the root directory. When it is executed it gets a listing of all directories in the posts directory and generates an index.html for each index.md file there. Then a table of contents is generated in the posts directory as index.html. Then it generates index.html in the root directory. Finally it iterates through a custom list for any directories where I may want it to generate additinal html files.

An an example of what the folder structure looks like.

learning.fail
    └── www
        ├── index.md
        ├── index.html
        └── posts
            ├── index.html
            └── example1
                ├── index.md
                ├── index.html
            └── example2
                ├── index.md
                ├── index.html
                ├── image.png
        └── style
            ├── bg.png
            ├── logo.png
            ├── main.css
    └── template
        ├── index.html
        ├── index.md
        ├── code.md
    ├── generate.py
    ├── README.md

All posts are expected to be in their own directory in the posts directory. Mainly so any assets for that post can be easily located and backed up. It has the added benifit of quickly adding or removeing posts and it's assets by simply copying or deleting a directory. All posts index.md also must have a title on the first line and a date on the third line. This is where the table of contents gets the title and date from.

An example from template/index.md

Title
=============
**MM/DD/YYYY**

The above text is required, in that order and that format. This and everything below is not. Such as this text.

Wrapping things up

All those things combined meet my primary goals. The site can easily be updated anywhere; ssh, ftp, desktop, etc. Does not require any specific editors and I don't have to worry about html. The rendering code is in python using only one external library, making it portable and simple to maintain. There is one primary issue I foresee with the framework, that is scalability. If I have a 1000 posts and directories it could take a long time for the site to render every time I make an edit. This can be solved by simply only rendering pages that have not been rendered or the markdown file is older then the html file.

generate.py

The code used to generate learning.fail. I didn't spend too much on it as I wanted to focuse on writing posts.

import markdown
import os
import time


class Config:
    def __init__(self):
        self.root_dir = "www"
        self.posts_dir = "posts"
        self.template_dir = "templates"
        self.pages = []

        self.template = "{}/index.html".format(self.template_dir)
        self.posts_dir = "{}/{}".format(self.root_dir, self.posts_dir)
        self.directory_list = os.listdir(self.posts_dir)
        self.post_list = [i for i in self.directory_list if os.path.isdir("{}/{}".format(self.posts_dir,i))]

        self.template_data = open(self.template, "r", encoding="utf-8").read()


class Posts():
    def __init__(self, post, config):
        self.post = post
        self.config = config


    def generate_post(self):
        path = "{}/{}".format(self.config.posts_dir, self.post)
        template_data = open(self.config.template, "r", encoding="utf-8").read()
        md_data = open(path + "/index.md", "r").read()
        html_data = convert_markdown(md_data)
        index = template_data.replace("{{main}}", html_data)
        open(path + "/index.html", "w").write(index)
        return True


    def generate_posts_table(self):
        md_data = open(self.config.posts_dir + "/index.md", "r").read()
        template_data = open(self.config.template, "r", encoding="utf-8").read()
        post_list = []

        for post in self.config.post_list:
            path = "{}/{}".format(self.config.posts_dir, post)
            post_data = open(path + "/index.md", "r").readlines()
            title = post_data[0].strip()
            created = int(time.mktime(time.strptime(post_data[2].strip().lstrip("**").rstrip("**"), "%m/%d/%Y")))

            for i in post_list:
                if i["created"] == created:
                    created += 1

            post_list.append({
                "created": created,
                "title": title,
                "post": post
                })

        post_list = sorted(post_list, key=lambda x: x['created'], reverse=True)

        for post in post_list:
            created = time.strftime("%m/%d/%Y", time.gmtime(post["created"]))
            md_data = md_data + "* [{}]({}) - {}\n".format(post["title"], post["post"], created)

        html_data = convert_markdown(md_data)
        index = template_data.replace("{{main}}", html_data)
        open(self.config.posts_dir + "/index.html", "w").write(index)
        return True


def convert_markdown(md_data):
    html = markdown.markdown(md_data, extensions=[
        'fenced_code',
        'codehilite'])

    return html


def generate_page(page, root_dir, template_data):

    try:
        if page == "index":
            md_data = open("{}/index.md".format(root_dir), "r").read()
        else:
            md_data = open("{}/{}/index.md".format(root_dir, page), "r").read()

    except:
        return False

    html_data = convert_markdown(md_data)
    index = template_data.replace("{{main}}", html_data)

    if page == "index":
        open("{}/index.html".format(root_dir), "w").write(index)
    else:
        open("{}/{}/index.html".format(root_dir, page), "w").write(index)

    return True


def main():
    config = Config()

    generate_page("index", config.root_dir, config.template_data)

    if config.pages:
        for i in config.pages:
            generate_page(i, config.root_dir, config.template)

    if config.post_list:
        for i in config.post_list:
            post = Posts(i, config)
            post.generate_post()

        post.generate_posts_table()


if __name__ == "__main__":
    main()

Thanks for reading,