Static Websites: Posting My First Article

This is the third article in a series about setting up my own website using a Static Site Generator. For other articles in the series, click on the title of the article under the heading “Static Site Generators” on the right side of the webpage.

Introduction¶

With Pelican installed, it was time to start creating stuff. The first I thing I wanted to do was to figure out if the base site was good by creating a default site and viewing it in a browser. Then I wanted to come up with a very simple test article, publishing it and viewing it to verify my ability to publish and exercise that workflow. Finally, I wanted to take some time to come up with a more representative test article and improve the publishing workflows.

Similar to the last article, this is not about leaping forward, but to take a decent step forward in understanding the tools and processes.

Before We Start¶

Please read Operating System Paths from the second article in this series for my view on operating system pathing. (i.e. /mount/my/path vs. c:\my\path). To understand the term base project directory and the script text %%%MY_DIRECTORY%%%, please read the explanation under Step 2: Create a Project Directory For The Site, also from the second article in this series.

Step 1: Verify Our Basic Web Site¶

The first thing I wanted to do is to make sure I didn’t mess anything up in the last article, Setting Up the Pelican Static Site Generator. Therefore, it made the most sense that I verify that my website was up and running before I added my first article. After doing some research, it was evident that there are two phases that I need to take care of: building the content and viewing the content.

Step 1a: Building The Web Site Content¶

The first article’s section on Static Site Generators (SSGs) hopefully made it clear that the focus of SSGs is to render the site content when updated. In the second article, Setting Up the Pelican Static Site Generator, I created the base configuration for a generic website, but did nothing to build that content. To build that content, I needed to find out how take the empty content and generate the output files from it.

Looking at some articles, I was drawn to the output of the pelican-quickstart command that generated the base configuration. In particular, I noticed the prompt:

> Do you want to generate a tasks.py/Makefile to automate generation and publishing? (Y/n) n

I specifically answered No because of my research. If you answer Yes, the template will ask a number of additional questions related to publishing, and it will place the Makefile and tasks.py in the website directory. As my primary machine is a Windows machine, these two files are mostly useless. Their primary value would be to instruct me on what I needed to do to replicate their behavior on the command line or in Python.

Thankfully, between the command pelican –help, using the Makefile for reference, and experimentation, I was able to arrive that this command for building the website:

pelican --debug --output website\output --settings website\pelicanconf.py website\content

Nothing too fancy. I started Pelican in debug mode to have more information about what was going on. Using the above command, the output was placed in the website\output directory, the settings were retrieved from the website\pelcianconf.py file, and the content for the website is located in the website\content directory. I double checked to make sure these were all correct, and it ran once without issue. To make sure that I didn’t lose this command, I created a simple script pelican-build.bat in my base project directory, with the above command being the only line in that file.

Pelican generated a lot of output, but in the end, it looked like everything was okay. I was excited to check out the new website, so I used my web browser to open the file website\output\index.html and got this result:

I must admit… it was kind of underwhelming. And then I thought about it a bit. A lot of the stuff that makes web pages powerful are other parts that included by the web page, and may not be loaded if the page is loaded directly from the file system.

To properly view the content, I was going to have to host it somewhere.

Step 1b: Viewing The Web Site Content¶

If you are like me, hearing or thinking the phrase “I need to install my own web server” fills me with read almost right away. Even for a pared down web server, there are usually 3-6 directories to set up, ports to clear with firewalls, and other little things. I must admit, when I started looking around, I was not exactly in a good mood.

However, the first 3 websites that talked about Pelican were from the Pelican project’s various versions. Looking deeper into the documentation, I found the part of the documentation titled Preview Your Site. Without taking too much away from the documentation, it specified that Python 3 includes a pared down web server that is available for simple uses, like I needed.

After a bit of experimentation and fiddling, and I came up with the following lines and placed them in pelican-server.bat:

pushd website\output
python -m pelican.server
popd

Executing that script, I then saw the following output:

XXX\python\python37-32\Lib\runpy.py:125: RuntimeWarning: 'pelican.server' found in sys.modules after import of package 'pelican', but prior to execution of 'pelican.server'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
WARNING: 'python -m pelican.server' is deprecated.
  | The Pelican development server should be run via 'pelican --listen' or 'pelican -l'.
  | This can be combined with regeneration as 'pelican -lr'.
  | Rerun 'pelican-quickstart' to get new Makefile and tasks.py files.
-> Serving at port 8000, server .
...

While this is what the official documentation suggests, it does look like it is out of date. Using the output from above, the installed Makefile for guidance, and more experimentation, I replaced the contents of the pelican-server.bat file with the following:

pelican -l -p 8000 --debug --output website\output --settings website\pelicanconf.py website\content

This time, the output of the script was:

DEBUG: Pelican version: 4.0.1
DEBUG: Python version: 3.7.3
DEBUG: Adding current directory to system path
DEBUG: Temporarily adding PLUGIN_PATHS to system path
DEBUG: Restoring system path
WARNING: Docutils has no localization for 'english'. Using 'en' instead.
-> Serving at port 8000, server .

To me, it looked like the web pages were served up properly, or at least Pelican didn’t report any errors to the screen. The only thing left was to actually see if it would allow me to load the generated website to my browser. Instead of loading the file directly into the web browser as before, I entered “localhost:8000” in the address bar. To be honest, I took a guess that server . meant the localhost.

This time, I got the following result:

Unstyled, no unique content, and only locally visible. It’s not a great result… but it’s a start!

Step 1c: Commit Changes¶

Before I went forward, I wanted to save the current state of the website, so back to Git and the git status -s command, whose output was now:

?? pelican-build.bat
?? pelican-server.bat
?? website/__pycache__/
?? website/output/

In my head, I compared these results to the changes I made:

• I added 2 script files to help me build and serve the content
• both file are detected, and I want them as source in my project
• I built the website, placing the output into website/output
• the directory was detected, but I want to generate this output, and not persist it
• from previous experience, the website/pycache/ contains pyc files that are built when the python scripts are interpreted
• the directory containing these files was detected, but these should never be persisted

Using thought processes derived from what I documented in the section Step 5: Committing the Changes from the previous article, it was clear that I needed to git add pelican-build.bat and git add pelican-server.bat to persist those changes. However, the two items that I did not want to persist would require different tacks for each one.

The first case, website/output/, is a directory like the virtualenv directory in the section # Step 2: Create a Project Directory For The Site from the previous article. Therefore, I edited the .gitignore file to include the directory by name. That was the simple one.

The second case was more complex, the website/pycache/ directory. This directory only exists to contain compiled Python byte code designed to speed up execution on subsequent passes. If I add the directory, as with the first case, it only takes care of that one directory. If I run any Python scripts from other locations, I will have to add those directories too. This was not an efficient option, hence I edited the .gitignore file to ignore the files themselves, by specifying &ast.pyc as the pattern to ignore.

Therefore, following those changes, the .gitignore file looked like:

virtualenv/
website/output/
*.pyc

Using git status -s, I verified that only the 2 scripts were being added, as well as the .gitignore file itself being changed. Quickly adding git add .gitignore, I then used git commit -m “my message” to commit these changes to the repository.

So let’s take inventory. We have a basic website up and running, we have tested it in our browser, and we have committed any changes to our local repository. It’s definitely time to write our first article.

Step 2: The First Article¶

Looking through the Pelican Documentation, I found another good section on Writing Content. From here, I learned that certain Markdown processors, such as the Python implementation, support metadata on files.

As a base example, they specify this as a sample post:

---
Title: My super title
Date: 2010-12-03 10:20
Modified: 2010-12-05 19:30
Category: Python
Tags: pelican, publishing
Slug: my-super-post
Authors: Alexis Metaireau, Conan Doyle
Summary: Short version for index and feeds
---

This is the content of my super blog post.

Reading further, a lot of the metadata fields have defaults, but the Title metadata is required for the page to be picked up and processed by Pelican. That got some wheels turning in my head, but I put it aside for a later article. First I wanted to have a solid way of writing the articles before adding to that.

Taking the markdown content from above, I created a file in the website/content directory called test.md and placed the content in there and saved it. Following the same pattern I used for the base content, I ran pelican-build.bat and then pelican-server.bat to view the content, providing the following:

Granted, its a stock article, but I now had a way to publish my articles! Along the way, I noticed a couple of things I noticed needing improvement.

Step 3a: A More Streamlined Build/Preview Workflow¶

In the previous section, the workflow I used took 4 actions to go from something I wrote to something I could see and validate: save the file, run pelican-build.bat, run pelican-server.bat, and refresh the web page. Wondering if there was a more efficient way to do that, I checked back on the Makefile that I used as the inspiration for the two scripts, and noticed a devserver entry.

As the devserver entry only differed from the server entry by the introduction of the -r flag, I quickly created a pelican-devserver.bat with that simple change, and executed it. Within seconds, I got the feedback:

CRITICAL: TypeError: can't pickle generator objects

Weird error, but understandable. In Python, the default serialization of objects to be passed outside of executables is referred to as pickling. If Pelican was already hosting the server and wanted to spawn a process to rebuild the website, it would make sense that it pass the current configuration to that new process.

Doing a bit more searching, I came across this issue logged against the Pelican project in GitHub. There are a fair number of entries for this issue, offering various pieces of advice. The low down for me is that due to the way pickling works, I cannot use it on my Windows machine with that given program. Reading further in the comments for the issue, a number of people did mention that the -r flag works fine when applied to the command I am using for building the pages, just not for building the pages from the server.

Trying it out, I renamed pelican-devserver.bat to pelican-autobuild.bat, and switched the file contents to:

pelican -r --output website\output --settings website\pelicanconf.py website\content

Based on what I read, this should have my desired effect, and a quick modification of the test.md file confirmed it. When I saved the file, the generator detected it and rebuilt the contents. However, thinking through things a bit more, I wondered if the Windows start command would help build a better solution.

When I write a Windows batch file and I need to execute another batch file, I use the call primitive. This allows me to run the new batch file within the current batch file’s shell. When that new batch file completes, I can check the return code from that script, and determine what action, if any, to do next. By structuring batch files and scripts this way, I find that I assemble things together more quickly, keeping the batch files and scripts more readable and more maintainable.

The start primitive is almost identical to the call primitive, with one small exception: the new program is started in a newly created shell spawned from the current shell. As the current shell and the new shell are separate processes, by default, both shells operate independently of each other, unless the /wait flag is specified. While this is not ideal for composing batch files, this behavior seemed to fit what I was trying to achieve.

Creating a new version of the pelican-devserver.bat file, I placed within it:

@echo off
start pelican-autobuild.bat
start pelican-server.bat

And executed the batch file. Two full size shell windows popped up, each one running one of the scripts, with the original shell window left alone. A new modification to the test.md file resulted in the rebuild happening in the window running pelican-autobuild.bat. A quick refresh of the browser page, and the newly changed article appeared in the browser.

I was mostly where I needed the script to be. Further tweaking the contents of pelican-devserver.bat to:

@echo off
start "Builder-Pelican" /min pelican-autobuild.bat
start "Server-Pelican" /min pelican-server.bat

got me to where I wanted to be. To keep things clean, I wanted to specify a title for each shell window and I wanted to start them minimized. Thus, by default both windows are out of the way, but if I need them, I can find them quickly.

Side Note: If you are using this as a guide, note that this only seems to be an issue on Windows machines. In quick tests I did using an Ubuntu image running under Docker, I did not see this issue, and the served pages updated properly.

Step 3b: A More Representative Test Article¶

The first thing I noticed about my test article is that it was short. Really short. When I posted it, it was hard to notice where the test article was being summarized, and where it was just the article. I needed to come up with a test article that would be a better representation of an actual article. For that, I dug back into my testing experience.

When coming up with test information for test projects, I have used the site Lorem Ipsum and sites like it for years. The site goes into more details, but by using this text it allows someone viewing the text to focus on what is around the text instead of the text itself. This is also better than using random words, as someone viewing that random text may try and find patterns in the paragraphs, once again distracting from what is around the text.

After a couple of quick clicks on the site, and it generated 5 decent sized paragraphs of random text that started with:

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc eget velit porta, efficitur justo at, sagittis nulla. Donec neque arcu, condimentum sed massa a, elementum rhoncus justo.
...

I didn’t include all 5 paragraphs here, as it was randomly generated. If you are using this as a guide, you can easily generate your own test paragraphs and insert them into your test message. However, when the page was regenerated, this is what it looked like:

Also, to give a bit of contrast, I created a copy of the page, called it test-2.md, changed the title a bit, and removed all except the first two paragraphs. My reasoning behind this was to make sure I had a sample that was long and a sample that was short.

Step 3c: Commit The Changes¶

As always, before moving on, I needed to make sure I committed the changes. This one was easy, simply performing the git add action on the files test.md, pelican-autobuild.bat, and pelican-devserver.bat, followed by a git commit.

What Was Accomplished¶

At the beginning of this article, I only had the foundation for a basic website. The first thing I did was to make sure I had a good workflow for generating the website and viewing it in a browser as a website. After that, I added a sample article, and also improved the original build/preview workflow. Finally, I created more realistic data for the test article, so I could see what a normal article would look like.

As for the goal of creating a test article that I could use to start fine-tuning the site, I believe I am in a good place. So next, fine-tuning and selecting a theme!

What’s Next?¶

Next, I need to clean up a couple of small things before selecting a theme for the website. Getting a solid choice for a theme is the last major task I have to complete before publishing the website itself. Completing this last major task will be covered in the next post Getting Ready For Publishing - Themes and Minutiae.

Like this post? Share on: Twitter ❄ Facebook ❄ Email