Turing School of Software Design: Module 2, Weeks 5 & 6

Little Shop of Orders is a Ruby on Rails project that requires small, 3-4 person teams of Turing students to build an online commerce platform to facilitate online ordering. My team's code can be found on our GitHub repository, and our application can be viewed here live. The project learning goals are to:

  • Use test driven development (TDD) to drive all layers of Rails development including unit and integration tests
  • Design a system of models which use one-to-one, one-to-many, and many-to-many relationships
  • Practice mixing HTML, CSS, and templates to create an inviting and usable User Interface
  • Differentiate responsibilities between components of the Rails stack
  • Build a logical user-flow that moves across multiple controllers and models
  • Practice an agile workflow and improve communication skills working within a team

To mimic an agile design shop, our instructors gave us batches of user stories through a project management collaboration tool called Waffle. We use Waffle because of its tight integration with GitHub, our source control platform. GitHub is kind of like Google Docs for coders because it allows multiple developers to work on a single project at the same time.

A user story for this project might read like:

The team got three rounds of user stories from our instructors, who played the role of the client in this project. Each round of user stories began with a team stand-up to review progress and decide next steps. A stand-up is an Agile practice which involves meeting participants actually standing up to meet. Standing up ensures that the meeting is quick and allows participants to get to work quickly. While we always had stand-ups after getting new batches of user stories, we found the practice so useful that we ended up running stand-ups at least once a day.

I like the concept so much, in fact, that I wish I had experienced this kind of meeting in the education world, but that might be one of the reasons I'm a recovering educator now. 

Test Driven Development

Once we agreed on the division of work related to a set of user stories, we got to work. Work always begins with writing a feature test, which is a translation of a user story into code. Using RSpec, a popular testing framework for Rails, we turned the above user story into the following feature test. 

require 'rails_helper'

RSpec.feature "admin can create items" do
  before(:each) do
    admin = Fabricate(:user, role: 1)
    @category = Fabricate(:category)
    login_as(admin)
    click_link "Create an Item"
  end

  scenario "admin can create item with photo attachment" do
    fill_in_title
    fill_in_description
    fill_in_price
    check_category
    attach_file("Image", Rails.root + "spec/fixtures/dummy.png")

    click_button "Create New Item"
    expect(current_path).to eq(item_path(Item.last))

    expect(page).to have_css(".product-image")
  end
end

It is easy to see why RSpec is so popular - the test reads almost the same as the user story. Lets take a look at what this code does one chunk at a time.

require 'rails_helper'

RSpec.feature "admin can create items" do
  before(:each) do
    admin = Fabricate(:user, role: 1)
    @category = Fabricate(:category)
    login_as(admin)
    click_link "Create an Item"
  end

The above line tells RSpec that our code should have a feature that allows an administrator to create an item. The string "admin can create items" is a label for the developers so that the when the test runs (and it is one of over a hundred tests in our project's test suite) it's easy to see which features are failing or passing.

Our project has a testing, development, and production environment. This best practice is used because as a developer I want to be able to run my application locally (development) on my own computer without it impacting my users, who use the production version of the application. In this way I can create new features and even break my application without losing business that comes with application crashes. The testing environment is important so that my testing suite does not affect the version of my application that I run locally.

We deliberately clear the entire test environment database before each individual test to make a more controlled environment for each test. Unexpected data can easily break or invalidate a test. From there, it's clear why it makes sense to run tests in a separate environment. Imagine if every time Amazon wanted to test a new button on their website, they had to delete every single user and product in their database. That is not a workable approach to testing. 

All that is to say that with RSpec, our feature tests often begin with some sort of database setup because every test begins with a blank slate.

admin = Fabricate(:user, role: 1)

In the above code we add a user with administrative privileges to our database, so at this point of our test, our entire store has one admin user without any items. Fabricate is a method from the Fabrication gem we have included in our project, which generate schematics for Ruby objects, and can be created as needed. When we say 'fabricate a user' in the above code, the Fabricate gem runs the following fabricator file:

# spec/fabricators/user_fabricator.rb
Fabricator(:user) do
  username { Faker::Name.name + (Faker::Number.between(1, 9)).to_s }
  password "password"
  user_profile
end

The documentation on the Fabricate website is pretty excellent. The above method is our schematic for a user, which has a username, password, and a user_profile. Users also have a role of 1 or 0, which we specified to be 1 for admin, but otherwise just defaults to 0 for default user. The Faker call in there is another gem that creates random text from various libraries of pre-created fake data. So Faker has a library of fake names and numbers, and in our item Fabricator, we ask Faker to supply a fake name and number that we use to create a username. The username is not important for the sake of our test because at the end of test the user is deleted anyway. What is important is that the user has a username.

@category = Fabricate(:category)

Moving back to the actual test file, we have to fabricate a category because our project specifications require that items have many categories. So the way we have our database and Item model defined means that in order to test how an admin creates an item, our database first must have a category. We use the following fabricator file to fabricate our category:

# spec/fabricators/category_fabricator.rb
Fabricator(:category) do
  name { Faker::Commerce.product_name }
end

Again, we use the Faker gem to create a random category name.

login_as(admin)
click_link "Create an Item"

Along with RSpec, we use a testing framework called Capybara, helps developers test web applications by simulating how a real user would interact with an application. So the above lines tell Capybara to simulate an admin loggin into the website and clicking a link called "Create an Item." That is the end of our before(:each) action, which runs at the start of every scenario in this test file. In other words, this is setting up our database before each test.

scenario "admin can create item with photo attachment" do
  fill_in_title
  fill_in_description
  fill_in_price
  check_category

In our scenario testing the photo upload and item creation feature, we continue to use Capybara to simulate the completion of a form that includes the title, description, price and category. These are all helper methods that we extracted into a /spec/helpers folder because they were actions we found ourselves using over and over. Here is an example of the fill_in_title method. It's nothing fancy, just some code extraction to keep things as readable and DRY (Don't Repeat Yourself) as possible.

# spec/helpers/helper_methods.rb
def fill_in_title
  fill_in "Title", with: "ItemTitle"
end

The next step in testing this feature is simulating the process of attaching a file in the form. Capybara comes loaded with exactly this type of functionality. 

attach_file("Image", Rails.root + "spec/fixtures/dummy.png")

To finish we tell Capybara to click "Create New Item."

click_button "Create New Item"

At this point in our test, we have had Capybara walk through the process of creating an item through an item creation form including the uploading of a photo. 

expect(current_path).to eq(item_path(Item.last))
expect(page).to have_css(".product-image")

Then we use RSpec to say that we expect the current_path, which is the page that the test admin ends up at once it clicks "Create New Item", to be the item path. The item path is the individual show page for a single item. We add a final expectation that the item show page include a product image. If the test code runs through this point, we know that an admin user can create an item with a picture. All that, and we still have zero functionality built. But, what we do have is a very clear road map of exactly what needs to get built.

At submission, our project ended up including 23 separate feature test files which test their own user stories.

one-to-one, one-to-many, and many-to-many relationships

For the above feature to be implemented, we need to consider the relationship the objects discussed in the user story have with one another. We know that we need to have an item object. We also need a user object. While not explicitly stated in the test, for the purposes of writing about how different objects relate to one another I'll also reference a few other objects, such as the order object and the user profile object that were described in other user stories. 

When we discuss database models we talk about how objects relate to each other. The most straightforward type of relationship is a one to one relationship. For example, each single user object has a user profile containing information like name and address. While our entire application may have multiple user and user profile objects, each user has one user profile that is unique. In Rails, to model this relationship we create a database migration that creates the following schema in our database. Notice that the user_profiles has an "add_index" pointing to the users table. That is adding a user id to every row of the user profiles table. This is called adding a foreign key because an id from one table is added to another table. The user id is added to the user profile table from the users table.

create_table "user_profiles", force: :cascade do |t|
  t.integer "user_id"
  t.string  "first_name"
  t.string  "last_name"
  t.string  "street_address"
  t.string  "city"
  t.string  "state"
  t.string  "zipcode"
end

add_index "user_profiles", ["user_id"], name: "index_user_profiles_on_user_id", using: :btree

create_table "users", force: :cascade do |t|
  t.string   "username"
  t.string   "password_digest"
  t.datetime "created_at",                  null: false
  t.datetime "updated_at",                  null: false
  t.integer  "role",            default: 0
end

We further elaborate the relationship with ActiveRecord in our app/models/user.rb file with the following. 

# user.rb
has_one :user_profile

# user_profile.rb
has_one :user

This gives us the ability to use ActiveRecord method calls, like user.user_profile and user_profile.user automatically just by having our models for user and user profile inherit from ActiveRecord::Base. This is the kind of functionality that I spent two weeks building manually over the course of the final project in Module 1, Black Thursday. Now I get it for free.

The next type of relationship our database will model is the one to many relationship. An example of this is that a user can place many orders (that's the whole point), but an order originates from a single user. 

create_table "orders", force: :cascade do |t|
  t.integer  "status",                              default: 0
  t.decimal  "total",      precision: 15, scale: 2
  t.integer  "user_id"
  t.datetime "created_at",                                      null: false
  t.datetime "updated_at",                                      null: false
end

add_index "orders", ["user_id"], name: "index_orders_on_user_id", using: :btree

In this code, notice again we are adding a foreign key from users to the orders table. Our models must also include the following methods to inherit the ActiveRecord functionality that allows us to call app/models/user.orders and app/models/order.users.

# user.rb
has_many :orders

# order.rb
belongs_to :user

Finally we have many to many relationships, which are slightly more tricky to model than the previous two because they typically require the creation of a join table. For example, an order can have many items, but items can also have many different orders. More plainly, as a user I should be able to order a stapler and some staples. But, the application also needs to allow a second user to order a stapler and a pen, or just some staples.

create_table "item_orders", force: :cascade do |t|
  t.integer "quantity"
  t.integer "order_id"
  t.integer "item_id"
end

add_index "item_orders", ["item_id"], name: "index_item_orders_on_item_id", using: :btree
add_index "item_orders", ["order_id"], name: "index_item_orders_on_order_id", using: :btree

create_table "items", force: :cascade do |t|
  t.string   "title"
  t.string   "description"
  t.string   "image_url"
  t.decimal  "price",              precision: 15, scale: 2
  t.datetime "created_at",                                                  null: false
  t.datetime "updated_at",                                                  null: false
  t.boolean  "retired",                                     default: false
  t.string   "image_file_name"
  t.string   "image_content_type"
  t.integer  "image_file_size"
  t.datetime "image_updated_at"
end

In this code, notice we have a third table, item_orders, which represents an item and quantity on a single order. We are adding a foreign key from items and orders to the item_orders table. By doing so and adding the following ActiveRecord methods in our models, we can call items from particular orders and orders that include particular items - order.items and item.orders. Thanks ActiveRecord.

# order.rb
has_many :item_orders
has_many :items, through: :item_orders

# item.rb
has_many :item_orders
has_many :orders, through: :item_orders

# item_order.rb
belongs_to :item
belongs_to :order

HTML, CSS, and templates

Going back to our original example of an administrator adding an item to the database with a photo attachment, the test should fail on the very first line which asks simulated administrator to fill in a title for the item in the item creation form. Before we can do that, we need to create an app/views/item/new.html.erb file with the following code.

<div class="col-md-6 col-md-offset-3 text-center">
  <h1>Create Item</h1>
  <%= form_for(@item, multipart: true) do |f| %>
    <div class="form-group text-left">
      <%= f.label :title %>
      <%= f.text_field :title, :class => "form-control" %>
    </div>

    <div class="form-group text-left">
      <%= f.label :description %>
      <%= f.text_area :description, :class => "form-control" %>
    </div>

    <div class="form-group text-left">
      <%= f.label :price %>
      <%= f.number_field :price, :class => "form-control", step: 'any' %>
    </div>

    <%= f.collection_check_boxes :category_ids, @categories, :id, :name do |cb| %>
      <% cb.label {cb.check_box + cb.text} %>
    <% end %>

    <div class="form-group text-left">
      <%= f.label :image %>
      <%= f.file_field :image, :class => "form-control" %>
    </div>

    <div class=btn-group>
      <%= f.submit "Create New Item", :class => "btn btn-primary" %>
    </div>
  <% end %>
</div>

A nice part of Rails are the various helpers it comes built with. So rather than coding an entire form in straight HTML, we can use the form_for helper method, which is less onerous. By applying a few simple Bootstrap classes and including the Bootstrap gem in our project, we get a professionally styled item creation form, with very little actual work. 

Models, Controllers and the Rails Stack

Next our test expects that when the "Create New Item" button is clicked, we will find ourselves on the newly created item's show page. To get there we'll use our item controller. This is a good moment to pause and just discuss the general idea of the model, view, controller (MVC). MVC is a software architectural pattern in which the view, controller, and model are all segregated into different files. The goal is to make it possible for a developer to make changes to one section without affecting the other two. The model is the portion of the code that interacts with the database. It is called the model because it is responsible for modeling objects in code. The view is the code that displays data to the user. The controller is the part of the code that takes a request from the user and responds with the appropriate view and data. So, maybe I want to change the style of the above new item form. I should be able to make that change without having to change the model or the controller.

The controller that handles requests created by the above form is as follows:

class ItemsController < ApplicationController

  def index
    @items = Item.all
  end

  def show
    @item = Item.find(params[:id])
    @review = Review.new
  end

  def new
    check_user_authorization
    @item = Item.new
    @categories = Category.all
  end

  def create
    redirect_to login_path if current_default_user? || !logged_in?
    @item = Item.new(item_params)
    assign_valid_category_ids(item_params, @item)
    check_if_item_can_be_saved(@item)
  end

  private

  def item_params
    params.require(:item).permit(:title, :description, :price, :image, :category_ids => [] )
  end

  def check_user_authorization
    if !logged_in?
      redirect_to login_path
    elsif current_default_user?
      render :file => "public/404.html", status: :not_found
    end
  end

  def assign_valid_category_ids(item_params, item)
    item_params['category_ids'].reject{ |id| id==''}.each do |id|
      @item.categories << Category.find(id)
    end
  end

  def check_if_item_can_be_saved(item)
    if @item.save
      flash[:success] = "Item was successfully saved."
      redirect_to item_path(@item)
    else
      flash[:danger] = "Item could not be saved."
      redirect_to new_item_path
    end
  end

end

The specific part of the controller that handles the creation of items via form submission is the create method:

def create
  redirect_to login_path if current_default_user? || !logged_in?
  @item = Item.new(item_params)
  assign_valid_category_ids(item_params, @item)
  check_if_item_can_be_saved(@item)
end

In this method we first do some authorization by checking to make sure that the user is logged in and is also an administrator. If either of those conditions are not met, we ignore the request to make a new item and redirect the user to the login screen.

If the request comes from an administrator, we create a new item and pass in the item_params hash.

def item_params
  params.require(:item).permit(:title, :description, :price, :image, :category_ids => [] )
end

The item_params hash is a feature built into Rails that allows developers to whitelist certain form submission parameters. What the code above is saying is basically when someone submits a form to create a new item, accept data from fields called "title", "description", "price", "image" and "category_ids". This is a security measure because we don't want to just take any arbitrary value that a malicious user might try to submit into our application.

We next add the newly created item to the selected categories.

def assign_valid_category_ids(item_params, item)
  item_params['category_ids'].reject{ |id| id==''}.each do |id|
    @item.categories << Category.find(id)
  end
end

We do one more check and test if the item can be saved. This shoots us down into the model of the item, where we validate what exactly constitutes an item. For example, an item must have a title, so any form submission that comes in without a title would be automatically invalid.

validates_presence_of :title, :description
validates_uniqueness_of :title
validates_numericality_of :price, greater_than: 0
validates_presence_of :categories

has_attached_file :image, styles: {
 thumb: '100x100>',
 square: '200x200#',
 medium: '300x300>'
}
validates_attachment_content_type :image, :content_type => /\Aimage\/.*\Z/

The above validations live in the app/models/item.rb file and check to make sure that an item has a title and description. The item must also have a unique title. It also must have a price greater than 0 and belong to a category.

The final two validations come from the Paperclip gem, which handles the photo upload. Paperclip takes a file attachment and allows us to interact with it the same way we would any other attribute of an object in the database. The validations test to make sure that the item has a file attachment and that the attachment must be an image.

Once an item is created, Paperclip will store the files in an Amazon Web Service (AWS) cloud storage bucket. This took a little finagling to get in order, but Heroku's documentation on how to set this up was quite helpful. A few key points that tripped us up along the way were:

  1. Setting the AWS cloud storage region is important. From the documentation we thought picking US-Standard would be fine, but we needed a more specific region.
  2. Getting an API key and Secret Key from AWS and storing that securely in the app was a little tricky. If anyone has access to your API key they can access your AWS account which contains credit card information. You don't want that information freely available on the internet. To prevent this from happening we used the Figaro gem, which helps store sensitive information privately in a Rails project.
  3. Finally, Paperclip cannot post data into an AWS cloud storage bucket until the user whose API credentials are being used by Paperclip has administrative rights to the storage bucket. Those privileges are not the default setting, so we had to add an administrative policy to the AWS user we created for this application.

Those were the three points that had us really stumped, so hopefully that helps other folks who are new to Paperclip. In general, Paperclip is remarkably straight forward and I want to give a shout out to ThoughtBot, which created the gem.

Once we got Paperclip working with AWS our original feature test passed and we saw that it was good.

Concluding Thoughts

Overall, this project was a great learning experience. This was a real exercise in synthesizing the ideas and techniques we had learned in various tutorials and lessons throughout Module 2. While there were definitely some challenges, the project never felt daunting in the sense that I doubted my ability to implement the required features - it really was in my proximal zone of development. 

I also had an amazing team that I hope I get to collaborate with in the future. So thank you Sonia, Susi, and David - you three were the best partners I could have asked for on this project. Folks who are looking to hire software developers this January, you could do a lot worse than these three coders, and I don't think you could do much better.