This is part 3 of this series. Go here for part 1 and here for part 2.
In part 2 of this series I laid out the foundation for this project as well as the issues and solutions I decided on. So now lets actually write some code.
go mod init foo
I first made a Github repo, cloned it locally and ran go mod init to start. I did some searching on directory structure and organization for an API with the intent to not end up with a monolith of a main.go file. Since I’ll be breaking this API into modules I settled on the following…
configs- Where I’ll create separate files to load the Mongo creds using godotenv and then create the actual connection to the db.controllers- The file here is where I’ll load collections and perform CRUD operations on them.models- Here is where I’ll specify the format of the data I expect to pull from the DB and also send to the DB.public- I can create HTML files here for echo to display.responses- The file here determines how responses are returned. They’ll include the HTTP code (e.g. 200), the message (e.g. Success) and a JSON response of the data if any.routes- This is where I’ll build my endpoints such as GET, PUT, etc.
Starting with this layout will make this API easier to maintain over time.
Hello Mongo
Since this API will query data from my Mongo DB we should probably start with connecting to it. We’ll establish both the loading/verification of the .env file and the DB connection in the configs dir and in different files. Lets start with loading the .env file.
To connect to Mongo we need to set a Mongo URI environment variable which is easy enough. We get our specific string from our Mongo instance, modify it if we’re using a user/pass or x509 cert and simply store the string in a file named .env.
With our Mongo connection string stored locally and not in code we need to confirm our .env file exists, that it contains data and then return that data to whatever calls it to establish a connection. We do so with the below code.
package configs
import (
"github.com/joho/godotenv"
"log"
"os"
)
func MongoURI() string {
if err := godotenv.Load(); err != nil {
log.Println("No .env file found")
}
uri := os.Getenv("MONGOURI")
if uri == "" {
log.Fatal("You must set your 'MONGOURI' environment variable. See\n\t https://www.mongodb.com/docs/drivers/go/current/usage-examples/#environment-variable")
}
return uri
}
This is fairly easy to grok; godotenv.Load() attempts to load a .env file and it if can’t find one it throws an error. If it finds the file then it sets the string inside it as an environment variable.
The uri variable reads in that environment variable. We then check the variable to make sure it’s not an empty string, if it is then we throw an error. If it’s not then we return that data to what called it.
Making the Connection
Now that we’re reading in our Mongo creds we need to establish a connection to it and we should also ping the DB to make sure we’re actually connected. We can achieve this with the below.
package configs
import (
"context"
"fmt"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"log"
"time"
)
func ConnectDB() *mongo.Client {
client, err := mongo.NewClient(options.Client().ApplyURI(MongoURI()))
if err != nil {
log.Fatal(err)
}
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
err = client.Connect(ctx)
if err != nil {
log.Fatal(err)
}
//ping the database
err = client.Ping(ctx, nil)
if err != nil {
log.Fatal(err)
}
fmt.Println("Connected to MongoDB")
return client
}
Let’s break it down a bit. First we create a client variable which creates a new mongo client which calls the MongoURI function we created earlier then it checks it for an error and stops if it finds one.
The ctx var creates a 10 second timeout for this connection and if it exceeds the 10 seconds it throws a timeout error.
Lastly, we ping the database and assuming we didn’t hit an error along the way we’re greeted with Connected to MongoDB.
Yeah I’ve Done Some Modeling
Before we go further we need to specify how data should be modeled. Specifically, I mean we should decide how we want our data to look when it’s called from the database and, more importantly, what format it should conform to when puts or updates are made via the API.
When we imported our data into MongoDB it automatically assigns it an ID in BSON (binary JSON) and in order to read or write this type we need to use a specific go import go.mongodb.org/mongo-driver/bson/primitive. This will be our only import in this package which is aptly named models and will, unsurprisingly, go into our models dir.
This code will be a series of struct types that mirror the row names in our database for each corresponding data type. Let’s look at some code…
package models
import "go.mongodb.org/mongo-driver/bson/primitive"
type Species struct {
Id primitive.ObjectID `bson:"_id" json:"_id,omitempty"`
Name string `json:"name" validate:"required"`
STR int `json:"str" validate:"required"`
AGI int `json:"agi" validate:"required"`
END int `json:"end" validate:"required"`
INT int `json:"int" validate:"required"`
LOG int `json:"log" validate:"required"`
WIL int `json:"wil" validate:"required"`
CHA int `json:"cha" validate:"required"`
REP int `json:"rep" validate:"required"`
LUC int `json:"luc" validate:"required"`
PSI_MAG_CHI int `json:"psi_mag_chi" validate:"required"`
Skill_Choices string `json:"skill_choices,omitempty" validate:"required"`
Exploits string `json:"exploits,omitempty" validate:"required"`
Age_Multiplier int `json:"age_multiplier,omitempty" validate:"required"`
Damage string `json:"damage,omitempty" validate:"required"`
Size string `json:"size,omitempty" validate:"required"`
Source string `json:"source,omitempty" validate:"required"`
}
type Origin struct {
Id primitive.ObjectID `bson:"_id" json:"_id,omitempty"`
Name string `json:"name" validate:"required"`
Prerequisite string `json:"prerequisite,omitempty" validate:"required"`
STR int `json:"str" validate:"required"`
AGI int `json:"agi" validate:"required"`
END int `json:"end" validate:"required"`
INT int `json:"int" validate:"required"`
LOG int `json:"log" validate:"required"`
WIL int `json:"wil" validate:"required"`
CHA int `json:"cha" validate:"required"`
REP int `json:"rep" validate:"required"`
LUC int `json:"luc" validate:"required"`
PSI_MAG_CHI int `json:"psi_mag_chi" validate:"required"`
Skill_Choices string `json:"skill_choices,omitempty" validate:"required"`
Exploits string `json:"exploits,omitempty" validate:"required"`
Years string `json:"size,omitempty" validate:"required"`
Source string `json:"source,omitempty" validate:"required"`
}
type Career struct {
Id primitive.ObjectID `bson:"_id" json:"_id,omitempty"`
Name string `json:"name" validate:"required"`
STR int `json:"str" validate:"required"`
AGI int `json:"agi" validate:"required"`
END int `json:"end" validate:"required"`
INT int `json:"int" validate:"required"`
LOG int `json:"log" validate:"required"`
WIL int `json:"wil" validate:"required"`
CHA int `json:"cha" validate:"required"`
REP int `json:"rep" validate:"required"`
LUC int `json:"luc" validate:"required"`
PSI_MAG_CHI int `json:"psi_mag_chi" validate:"required"`
Prerequisites string `json:"prerequisites,omitempty" validate:"required"`
Skill_Choices string `json:"skill_choices,omitempty" validate:"required"`
Exploits string `json:"exploits,omitempty" validate:"required"`
Years string `json:"size,omitempty" validate:"required"`
Source string `json:"source,omitempty" validate:"required"`
}
type UniversalExploit struct {
Id primitive.ObjectID `bson:"_id" json:"_id,omitempty"`
Name string `json:"name" validate:"required"`
Type string `json:"type,omitempty" validate:"required"`
Description string `json:"description,omitempty" validate:"required"`
}
type CrExploit struct {
Id primitive.ObjectID `bson:"_id" json:"_id,omitempty"`
Name string `json:"name" validate:"required"`
Source string `json:"source,omitempty" validate:"required"`
Description string `json:"description,omitempty" validate:"required"`
Prerequisites string `json:"prerequisites,omitempty" validate:"required"`
}
So I’ve created a struct with how I want each call (GET, PUT, UPDATE) to look for each corresponding collection in the DB. The columns are pretty repetitive for each struct in this case so I won’t explain each row. This is a happy accident in this project as depending on what your API is for there may be many many more.
Reading from left to right we first start with Id which is a primitive ObjectID type and we specify two acceptable data types, either bson or json and in the case of JSON we’ve added omitempty. The reason I added this is perhaps I want to PUT data into the Mongo DB using the API. Since Mongo automatically assigns the object ID we can safely omit this information in the event we’re doing a PUT. Other data types we’re using here are string and int but others exist of course.
Public
This is the directory where I’ll setup my stretch goal hopefully. It’s the directory where I’ll put publicaly accessible site files. That HTML class I took in college is really paying off and I created a basic index.html file here which just points towards the main WOIN site.
<html>
<p>Check out <a href="http://www.woinrpg.com">WOIN</a></p>
</html>
The stretch goal is to add a webapp for character creation and documentation for the API.
Responses
This file is pretty simple. All I’m doing here is declaring how responses to API calls are structured. I’m using a struct to ensure that every response returns an http status like 200, 404, 503, a message like Success or Failure and the data returned from the call to the DB.
package responses
import (
"github.com/labstack/echo/v4"
)
type Response struct {
Status int `json:"status"`
Message string `json:"message"`
Data *echo.Map `json:"data"`
}
Here is an example of a GET response…
{"status":200,"message":"success","data":{"species":{"_id":"6305358afc82b7f384fda793","name":"Borian","str":0,"agi":0,"end":1,"int":1,"log":0,"wil":0,"cha":1,"rep":0,"luc":0,"psi_mag_chi":0,"skill_choices":"carousing, hardy, [crafting], engineering, appraisal","exploits":"Darksight, Iron Constitution, Tinkerer, Long-lived, Personable, Small","age_multiplier":3,"damage":"1d6","size":"small","source":"NEW"}}}
Routes
Here I’m defining where API calls go. I’m only implementing GET routes at this time because I’m still learning how to build an API correctly and don’t want to risk opening myself up to a bad actor by having PUT, UPDATE or DELETE abilities for now.
Here’s an example of the base (root) route and what it looks like to GET a species or all species in WOIN.
package routes
import (
"github.com/labstack/echo/v4"
"woin-api/controllers"
)
func BaseRoute(e *echo.Echo) {
e.GET("/", controllers.GetBase)
}
func SpeciesRoute(e *echo.Echo) {
e.GET("/v1/species/:speciesName", controllers.GetSpecies)
e.GET("/v1/species", controllers.GetAllSpecies)
}
Notice the routes contain paths that will exist in my url as well as a version, in this case v1. I’ve included a version because over time I expect to improve on this and may overhaul it entirely once someone reads this blog and informs me it’s a steaming pile of trash.
You’ll also notice in the imports section that I’m using a controllers module to help with the routes. This file imports our other modules as well as loading in the Mongo collections. I’m then using functions to make the GET calls, for example, here is the function to GET a single species:
func GetSpecies(c echo.Context) error {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
speciesName := c.Param("speciesName")
var species models.Species
defer cancel()
err := speciesCollection.FindOne(ctx, bson.M{"name": speciesName}).Decode(&species)
if err != nil {
return c.JSON(http.StatusInternalServerError, responses.Response{Status: http.StatusInternalServerError, Message: "error", Data: &echo.Map{"data": err.Error()}})
}
return c.JSON(http.StatusOK, responses.Response{Status: http.StatusOK, Message: "success", Data: &echo.Map{"species": species}})
}
We start by setting a context and cancel var which essentially primes our call to Mongo and gives it a 10 second timeout. Next give our species name a variable and then pass our species data model to the species variable. Then we defer our cancel.
err is then assigned to our Mongo call and we move to our if statement. If our call to Mongo succeded (i.e. not nil) then we move to our return statement. If it’s nil and the call failed we return an error response.
This process is essentially the same when you make a GET request to retrieve all of something, like species. See below…
func GetAllSpecies(c echo.Context) error {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
var species []models.Species
defer cancel()
results, err := speciesCollection.Find(ctx, bson.M{})
if err != nil {
return c.JSON(http.StatusInternalServerError, responses.Response{Status: http.StatusInternalServerError, Message: "error", Data: &echo.Map{"data": err.Error()}})
}
defer results.Close(ctx)
for results.Next(ctx) {
var singleSpecies models.Species
if err = results.Decode(&singleSpecies); err != nil {
return c.JSON(http.StatusInternalServerError, responses.Response{Status: http.StatusInternalServerError, Message: "error", Data: &echo.Map{"data": err.Error()}})
}
species = append(species, singleSpecies)
}
return c.JSON(http.StatusOK, responses.Response{Status: http.StatusOK, Message: "success", Data: &echo.Map{"data": species}})
}
Here we’re doing a Find instead of FindOne in our call to Mongo. Then we’re taking each result (or species here) and formatting the data to the model and appending it into what tends to be a very large JSON result.
Conclusion
So that’s a mostly terse explanation of how this works. I plan to make more improvements over time like adding in a secure way for me to do PUT/DELETE/UPDATE calls, better improve errors since regardless of the type of error they all come back a 500 internal server error and in time, hopefully a web app that can utilize this api to create WOIN characters.
In part 4 of this series I’ll conclude with putting this API on the AWS lightsail instance, setting it up as a auto-starting app in systemd, getting ssl enabled and a url so that calls can be made. Tune in next time ;)