Scott Watermasysk

Hash#new

Mon, 25 Mar 2019 13:32:30 +0000

I was reading this post on Hash#fetch, and it reminded me of another lesser used/understood method on Ruby’s Hash object, new.

Typically, in Ruby, a hash is created by using just the {} literal. This is the equivalent of just doing Hash.new.

h1 = {}
h2 = Hash.new

Regardless of which option you choose, you get the same result.

However, as in most things Ruby, there is more than one way. The Hash initializer has three options:

No parameters
A default parameter
A Block

Next to the literal, the one I use most often is option #2, a default parameter. Typically, I use this when I want to count a bunch of things.

a = [1,2,3,4,1,1,3]
h = Hash.new(0)
a.each {|i| h[i] += 1}
puts h #{1=>3, 2=>1, 3=>2, 4=>1}

Without the Hash.new(0) we would have to check for nil, or do an assignment like (h[i] ||= 0) += 1 which is much less readable and depending on what you are iterating over can get complicated.

Multiple Heroku Environments

Mon, 25 Mar 2019 13:11:13 +0000

When you have multiple environments for an app on Heroku each command you execute requires you to pass in the -a (–app) flag with the app name. Heroku app names need to be unique not just for you, but for Heroku as a whole. This means the app name might not always be as memorable as you would like. KickoffLabs is actually broken up into three separate apps which further complicates things (6 total app names).

Instead of using the -a flag, you can also use the git remote name via the –remote flag.

The remote name only needs to be unique within your app. This means you can reign in this craziness a bit by using your own git remote names. By default, Heroku creates a remote called heroku. You can use the git remote rename command to rename your remote(s) to something more consistent

git remote rename original_name new_name

With this knowledge, you can create more consistent names for your remotes. I generally go with production and staging.

Next, I add the following to my zsh profile:

function hs(){
  heroku  "$@" --remote staging
}

function hp(){
  heroku  "$@" --remote production
}

Now, anything I want to do to production, is as simple as:

hp config:set VAR=1234

And the same can be done in staging:

hs config:set VAR=1234

Introducing Jekyll Rest

Mon, 11 Mar 2019 16:30:52 +0000

Executive Summary - Jekyll Rest allows you to publish to a Jekyll based site which uses Github as a repository.

One of the things I like about the Jekyll + Github + Netlify combination is there is nothing to manage. No backups. No (major) concerns with hosting. Ultimately, nothing to do but write¹. This is actual serverless IMO.

The trade-off is a slower publishing flow and usually being tied to a computer with Git access. Most of the time this is not as big of an issue as it would seem (how many of you blog more than once a week). However, I am continuing to try and flush out my “shorts” concept and always needing to be at a computer with Git access was slowing me down.

I had initially hoped I could leverage Zapier and avoid writing any code. Unfortunately, Zapier² only got me about 50% there. Zapier is willing and able to receive a post from me from just about any tool (web, email, and even SMS), but I could not figure out how to correctly format my content to make it work with the Github V3 rest API³.

The more I play with Zapier, the more convinced I am I could make it work, but I soon realized that sending in posts from various devices require a bit more finessing of the data. This ultimately made writing it in Ruby even more appealing.

In the end, I built something I am calling Jekyll Rest. It is a single endpoint you host on your own in a matter of seconds on Heroku.

It has just one endpoint, create. This endpoint was written to be as flexible as possible. You can specify all of the individual parameters (title, date, body, etc.) or you can send it a block of frontmatter. It will do its best to decipher what you meant, even extracting a title from your content if necessary. The rules for this are defined on the GitHub readme.

With this endpoint, you can do a couple of things:

You can add it Zapier. This enables you to compose blog posts via email or SMS.
Automate your workflow.

I have done both. I have it set up now so that I quickly email and txt new short posts. I also built a simple UI that helps me share various things I find interesting.

I will someday how I can see blogging as just writing ↩︎
Zapier is amazing. ↩︎
The biggest issue was base64 encoding the content ↩︎

Proxying Requests To Heroku Via Caddy Server

Thu, 07 Mar 2019 19:59:26 +0000

I wanted to experiment with using Caddy Server as a reverse proxy to an application hosted on Heroku.

If you are not familiar with Caddy, it is a newish, lightweight web server written in Go that was built from the ground up to leverage HTTPS (via Let’s Encrypt). Caddy is general purpose web server, but for now, I am only interested in using it as a reverse proxy.

You can create a minimalist version of a reverse proxy by adding a Caddyfile with the following:

yourdomainname.com
proxy / proxieddomain.com

You then start Caddy¹, and your requests from yourdomainname.com should be proxied to proxieddomain.com.

However, this does not work as is for Heroku. When a request is proxied to Heroku, it needs to know which app to direct the request to. Chances are, yourdomainname.com is not registered on Heroku².

As you might expect, Caddy allows you to specify some headers which are passed along to the server receiving the proxy request.

Even better, Caddy has a convenient helper, transparent³, which wraps up the standard headers.

Using transparent, our original example now looks like this:

yourdomainname.com
proxy / proxieddomain.com {
  transparent
}

To keep you up to speed, transparent is shorthand for:

yourdomainname.com
proxy / proxieddomain.com {
  header_upstream Host {host}
  header_upstream X-Real-IP {remote}
  header_upstream X-Forwarded-For {remote}
  header_upstream X-Forwarded-Port {server_port}
  header_upstream X-Forwarded-Proto {scheme}
}

Less Code == More Gooder! Well, except here. Heroku fails in the Host header is set to anything other than the proxieddomain.com. This one took quite a while to figure out. Thank you very much curl!

I reached out to Heroku and they confirmed that the HOST header is used internally

The Heroku router uses the host header for routing traffic on the platform. If you are specifying a host that is not setup as a custom domain on your application these sorts of requests will fail because they won’t be able to be routed.

Using header_upstream Host {host} the host header is set to yourdomainname.com⁴.

But in the end, the following works and provides all the benefits of transparent, just a couple of extra lines:

yourdomainname.com
proxy / proxieddomain.com {
  header_upstream X-Real-IP {remote}
  header_upstream X-Forwarded-For {remote}
  header_upstream X-Forwarded-Port {server_port}
  header_upstream X-Forwarded-Proto {scheme}
}

You will also likely want to include either header_upstream X-Forwarded-Host {host} or header_upstream Origin {host} so that you know which domain is actually being proxied.

If you are completely new, checkout the getting started guide to make sure you generate your SSL cert first ↩︎
Otherwise, why would you need to proxy it? 😀 ↩︎
“Passes thru host information from the original request as most backend apps would expect.” ↩︎
The {} in the configuration are Caddy’s placeholders. ↩︎

Bugs are Inevitable. Your Response is What Matters

Wed, 20 Feb 2019 13:22:29 +0000

Bug, defects, delays, etc will always happen. With good practices, you can minimize them, but they will always exist.

You can however control how you respond to them. This message from WASD is exceptional.¹

Accept fault
Apologize
Provide details and supporting information.

If you are following my keyboard journey, interactions with WASD like this won me over. ↩︎

Better Jekyll Excerpts

Tue, 12 Feb 2019 11:49:54 +0000

When you use the built-in excerpt feature, Jekyll returns the first paragraph of text. It does this by looking for something called the excerpt_separator. The excerpt_separator defaults to \n\n, which is ideal for most short posts.

Back in my LiveWriter days, it would use an HTML comment like  to designate where to break for an excerpt. Jekyll supports this as well. In your _config.yml file you can add:

excerpt_separator: <!--more-->

Unfortunately, this now means that on every post, you need to enter , even if you are OK with the default of the first paragraph. The alternative is to add excerpt_separator:  for each post you want to control the excerpt.

Adding the excerpt_separator when needed is slightly better, but still too manual for my tastes.

Instead, I wrote a small plugin you can drop in your _plugins directory.

Jekyll::Hooks.register :posts, :pre_render do |post|

  if post.data['excerpt_separator'].nil? && post.content =~ /^(<!--\s*more\s*-->)$/
    post.data["excerpt_separator"] = $1
    post.data['excerpt'] = Jekyll::Excerpt.new(post)
  end
  
end

For each post, when building, it looks to see if there is a  in the body. If it is found it sets it as the excerpt_separator. Otherwise, it uses the default \n\n.

Michelle Obama On Startup Pricing

Sun, 10 Feb 2019 12:53:54 +0000

On startup pricing, Michelle Obama famously said,

…our motto is: when they go low, we go high¹

A post on Indie Hackers reminded me of this quote:

So, I’m thinking if I should try offering an email service, similar to Google Apps, but more affordable..

Just raise prices has been the rallying cry in the bootstrap community for the last couple of years², but this is a different topic.

When competing with large entrenched competitors competing on price is almost always the wrong strategy. Founders, especially those who were developers, grossly underestimate what it takes to build and support something.

Google could easily run Google Apps for nothing (they did for a long time). The price is already low, and they likely could go lower. They have resources and margins you will never attain.

Instead, how you compete with an entrenched competitor is with things they cannot possibly compete with like support and personal interactions. In the case of Google Apps, a couple of dollars a month (even 100s) is not going to make a big difference. But personally setting up the DNS? Helping the customer figure out DKIM records, SPF records, correctly using their business domain for email with their other third-party integrations will make a significant difference and one that they will likely be able to pay for with much better margins.

Find your competitive advantage and use it.

According to the facts, I am taking this quote completely out of context. Please do not let the facts get in the way of my story. ↩︎
I am increasingly less convinced high(er) prices is the only answer. More on this in the future. ↩︎

Unlimited Software Does Not Exist

Wed, 06 Feb 2019 16:00:16 +0000

In software, nothing is unlimited. You can market and sell your product without any limits, but when it comes to computing, there are always limits.

One such limit, bit me in the ass yesterday.

KickoffLabs uses recursive queries in a couple of places. These are great because they can often limit hops back and forth to the database. Generally, they recurse two or so levels deep.

Unfortunately, a customer (or customer’s customer) managed to go many levels deep….and you probably guessed it by now, without any limits. Furthermore, one of these queries generates more related recursive queries.

The green in the picture is database work. It was not the number of queries causing the problem, just the amount of work they were doing. These queries in turn started to push everything else back, exhaust connections, etc….a fun time for all.

The good news is overall the system held up quite well. For a couple of hours our response time was over our desired limits, but we were still very much operational.

We now have a limit in place on the depth of our recursive queries.

These bugs frustrate the bajeebus¹ out of me. It is so obvious once you see it and it I borderline preach always having some limits.

As penance for my sins against computing, I could not sleep last night, and I ended up starting on the real fix at 2 am.

If you have a moment, look around your code and see if there is something which needs some a ‘just in case’ limit.

I got to use the word bajeebus…it is all worth it. ↩︎

What Are Shorts? And Why You Should Use Them Too.

Mon, 04 Feb 2019 23:23:07 +0000

There are few things lamer than blogging about blogging, but this is a subject I think is very important.

In my virtual circles, Twitter has killed blogging. I do not believe Twitter did it on purpose. Twitter is simple, and you can reach a lot of people quickly. And despite all the negative energy around Twitter, it still the place where I learn the most on a daily basis.

What concerns me about blogging’s demise is how much information vanishes after a couple of minutes. You can like it, but good luck finding in the future.

What I am trying to do instead is something I am calling shorts. A short is a post that is either small enough to live inside a tweet or has enough formatting applied so that it can start as a tweet and can be followed on site if it piques your interest.

I even went as far as building a simple UI which posts to this (static!) blog:

If you are interested in the logic around the posts, you can see my Jekyll hook here. In a nutshell:

Figure out if there is an excerpt defined or not
Format the excerpt/content
If it is a link to another site, link to it directly otherwise link back to this site only if we need more than 280 characters.

I then push the posts to their own feed and allow Zapier to deliver them to Twitter.

The other significant benefit of this approach is I will have a list of content that I find essential that lives outside of Twitter.

Long term, I would love to make this easy for everyone, but for now, I wanted to share how I am doing it and provide some tips and strategies for doing it yourself as well.

Simplifying Shorti's Api

Sat, 02 Feb 2019 14:24:46 +0000

If you are not familiar, Shorti is my API only simple URL shortener. The source is on GitHub.

Initially, all of the API endpoints returned JSON. JSON is convenient, and the tools around it are great these days. However, I did notice that in my primary use cases it was adding some unnecessary steps, especially when using it with shell scripts.

Since it is a relatively new project, I thought it was best to fix this now.

I made a change that now checks the HTTP ACCEPT header. If it is application/json, the API will return JSON. If it is anything else, it will return plain text.

  def is_json_request?
    request.accept == "application/json"
  end

What is nice is that you can now simply do something like this:

 curl -d "url=https://scottw.com&api_key=HOAGIES" https://shortilinks.herokuapp.com | pipe_some_where

Also of interest, respond_to is not available by default in a Rails API project. I am sure there is a way to retroactively add it, but for now I went with something simple.