Technologies Behind Caches To Caches

10 Aug 2015 Gregory J. Stein
Category Web Development

There are a number of different technologies which have gone into creating this blog. Now that it’s mostly complete (though I’m constantly tweaking its look), I figure I should give a list of these different tools so that someone can use it as a starting point for their own site. Though I’ve created a blog, many of the steps I’ve gone through are useful for anyone creating a small, content-driven website of their own.

In short, this site (and my personal website) are hosted on a single Amazon T2.micro instance using Django and Apache and wrote my own custom formatting with Less. However, there’s much more to it than that. I also made my mark in social media, being sure to get the pages associated with my site.

While I used to use Bootstrap for styling, I ultimately wanted more customization to the appearance of the site and it was easier to start from scratch. I discuss some of my design decisions in another one of my posts.

See more details after the jump.

Server Backend

As I stated above, I host this blog using the Amazon Web Services on Amazon T2.micro instance which should be more than sufficient, in terms of bandwidth and computation, for the foreseeable future. In addition, since I’m within my first year of the service, I’m eligible for the AWS Free Tier, so that my T2.micro costs me nothing.

The server is written entirely in python using the popular and powerful Django web framework. After looking into other options, like Ruby on Rails, I found Django had add-ons for everything I thought I may need and easy to use, since I use python frequently for other projects. In addition, Apache can directly run Django applications through the built-in mod_wsgi, which allows for communication between python applications and Apache. Also, Apache allows me to run multiple Django applications simultaneously, which means that I can host both this blog and my personal website on a single machine. More recently (as of early 2018), I have been using Docker with Docker Compose for both my development and production environments. Using Docker makes it easy to run a local version of my site when I’m changing things, yet is designed so that I can push my changes to my server with ease, without having to worry about what packages are installed on each machine.

Content and Images

I rely on Django to store my website’s content, which means that I (rather lazily) use the Django admin site to write blog posts and to handle the remainder of the content on my blog. However, it was a decent amount of work to get my python classes, known in Django as models, the way I liked them, so that I could have tags, categories and series associated with every blog post, and so that each of these labels could have their own cover image and extended descriptions.

I use the python Markdown implementation (which can be easily installed through pip) to process my blog post content, which is otherwise written in plain text. In addition, I have custom escape sequences in the text which are processed before the markdown, so that I can inject additional HTML code without having to modify the Django templates directly from the admin site. For example, whenever I save an image to the server through a post on the admin site, I can use my custom img escape command to search for the file location on server, rather than directly figuring out the URL to access it myself.

With regard to images, I save them to the server using Django’s ImageField model, and then serve them up using apache. In addition, since Django does not automatically delete saved images, I created a separate Image model class which takes care of this (otherwise missing) functionality. Furthermore, I rely on PIL, the ‘python image library’, to create thumbnails for each of the same images and to handle deletion of those files as well.

Third, I decided against building my own comment engine, since creating it would have been a hassle and policing it without additional tools would have been complicated. Instead, I decided to outsource this to Disqus, a popular comment hosting service. Though this means that I’ve sacrificed the ability to completely customize the look of the comment section, the ease of use of Disqus outweighs any other disadvantages.

Finally, as a member of the academic community, much of my worth is measured by my publication list. As such, my personal website has a section devoted to listing my publications. To do this, I use the fantastic django-publications framework. It’s built in BibTeX parser is enormously helpful: to the point that adding new publications from the server requires virtually no effort.

Layout

Formatting the blog was not particularly difficult, since I use Boostrap to do most of the heavy lifting. However, I really wanted the blog to look a specific way so I customized the look using the Less CSS preprocessor. If you’ve never used Less, I’d highly recommend it. It really adds functionality that I feel is otherwise lacking when I’m directly programming CSS, particularly the ability to define variables, which makes it extremely easy to try out different color and formatting schemes.

In addition to using Bootstrap for CSS, I rely on a number of its built-in javascript plugins, which naturally require jQuery. I use the dropdown functionality for the navigation bar at the top of the page and the scrollspy for the sidebar (which you can only see when you’re on a large enough device). In addition, I use a table of contents plugin to add items to the sidebar. I use the table of contents plugin to automatically populate the sidebar with the headings from the body of all the post. I chose this one, by Doug Neiner, since it does almost exactly what I wanted and nothing more, as opposed to tocify which is much less lightweight. Finally, I use the tooltip functionality for some of the post content and sidebar links, so that a short description shows up when the link is hovered over.

The sidebar is also “sticky”, meaning that it stays at the top of the page as its scrolled past. To do this, I modified some code from this StackOverflow post, which was limited in that it did not stop when the footer reached the bottom. My code, shown below, does not suffer from this limitation:

Sticky Sidebar JavaScript
$(function(){

  var above = '#sticky-top';
  var sticky = '#sticky-sidebar';
  var footer = '#footer';
  var vertOffset = 100;
  var footerPadding = 00;
  
  if (!!$(sticky).offset()) { // ensure sticky element exists

    var stickyTopInit = $(above).offset().top;

    $(window).scroll(function(){ // scroll event

      var windowTop = $(window).scrollTop() + vertOffset;
      var footerTop = $(footer).offset().top-footerPadding;
      var stickyTop = $(above).offset().top + $(above).height();

      if (stickyTop+$(sticky).height() > footerTop - vertOffset) {
        // Rare case when sidebar is supporting footer
        return
      }
    
      if (windowTop+$(sticky).height() >= footerTop) {
        // If the sidebar is below the footer, raise it up
        $(sticky).css({position:'relative',
                       top: footerTop-stickyTop-$(sticky).height(),
                       width: ''});
      } 
      else {
        if ( stickyTop < windowTop ) {
          // If the top of the window is below the sidebar, lower it
          // This will also return the sidebar to its position
          $(sticky).css({ position: 'fixed', top: vertOffset, width: $(above).width() });
        } else {
          // Return the sidebar to its correct place
          $(sticky).css({ position: 'static', width: '' });
        }
      }
    });
  }
});

Additional Resources

Beyond the tools listed above, there were a number of others that I used for the blog and, relatedly, increasing the presence of the blog on the web. Here are some of those resources that I found to be most useful.

Structured Data/Analytics [Google]

Behind the scenes, there was quite a lot I had to do in the way of measuring traffic to this site and ensuring search engines properly crawled the blog. I was not quite experienced enough to check that my search engine presence was adequate, and I relied on Google’s set of (free!) tools to get everything up and running. You may have noticed that I run Google Analytics to get an idea of how many people have visited Caches To Caches.

First, I did purchase my domains through Google Domains, which meant that they were automatically visible to the remainder of the tools without much additional effort. However, there are good instructions to add your site through Google Webmaster Tools. Using Google Webmaster Tools is extremely useful for making sure that Google actively indexes your site, so that it will appear on its search results, after a refresh period of up to a week. In addition, it will alert you if it encounters any errors during the crawl. Relatedly, Google’s provided Structured Data Testing Tool can help you ensure that your schema.org structured markup has been done correctly (which can be a real pain the first time around). Try entering the URL for this page on their interface to see how my content is structured.

I also have a Google Custom Search Engine, which returns Google search results limited to this site from the input field in the sidebar. It’s extremely easy to setup and use, and has a variety of visualization options for the results. In addition, the custom search engine can be instructed to rapidly crawl and index specific pages on my site, which made it an invaluable tool for ensuring that my search engine meta tags were properly set up.

Emacs

Emacs is handily my text/code editor of choice. After years of periodically coming back to it, I now use it for everything from my HTML and python editing, to actually writing the blog posts and taking notes about ideas for additional content. I won’t go into too much detail here, but if you’d like to learn more, I have a growing list of posts about it at the Emacs tag page.

Social Media

While not directly involved with the content on the blog, I did go out of my way to ensure that I pulled out all of the stops regarding social media. While I mostly did this for the experience, in case I were ever to create a product website in the future (perhaps a far off goal of mine), I recognize that it’s rather important in today’s culture that readers have both the ability to find a particular site through the most heavily trafficked social media sites and to share content through them. I created pages for Facebook, Twitter, Google+, and GitHub, in which I store much of my sample code.

Conclusion

I may continue to add-to/update this list as the blog continues to grow and change. For now, this list is mostly complete. Let me know if you think I’ve made any unwise decisions in the comments. I appreciate any suggestions.