Subscribe

Ten Useful Techniques in Go

Ten Useful Techniques in Go

Here are my own best practices that I’ve gathered from my personal experiences with dealing lots of Go code for the past years. I believe they all scale well. With scaling I mean:

  1. Your applications requirements are changing in an agile environment. You don’t want to refactor every piece of it after 3–4 months just because you need to. New features should be added easily.
  2. Your application is developed by many people, it should readable and easy to maintain.
  3. Your application is used by a lot of people, there will be bugs which should be found easily and fixed quickly

With time I’ve learned these things are important in long-term. Some of them are minor, but they affect a lot of things. These are all advice, try to adapt them and let me know if it works out for you. Feel free to comment :)

1. Use a single GOPATH

Multiple GOPATH’s doesn’t scale well. GOPATH itself is highly self-contained by nature (via import paths). Having multiple GOPATH’s can have side effects such as using a different version for a given package. You might have updated it in one place, but not in another. Having said that, I haven’t encountered a single case where multiple GOPATH’s are needed. Just use a single GOPATH and it will boost your Go development process.

I need to clarify one thing that come up and lots of people disagree with this statement. Big projects like etcd or camlistore are using vendoring trough freezing the dependencies to a folder with a tool like godep. That means those projects have a single GOPATH in their whole universe. They only see the versions that are available inside that vendor folder. Using different GOPATH for every single project is just an overkill unless you think your project is big and important one. If you think that your project needs its own GOPATH folder go and create one, however until that time don’t try to use multiple GOPATH’s. It will just slow down you.

2. Wrap for-select idiom to a function

If there is a situation where you need to break out of from a for-select idiom, you need to use labels. An example would be:

func main() {

L:
    for {
        select {
        case <-time.After(time.Second):
            fmt.Println("hello")
        default:
            break L
        }
    }

    fmt.Println("ending")
}

As you see you need use break in conjunction with a label. This has his place, but I don’t like it. The for loop seems to be small in our example, but usually its much more larger and tracking the state of break is tedious.

I’m wrapping for-select idioms into a function if I need to break out:

func main() {
    foo()
    fmt.Println("ending")
}

func foo() {
    for {
        select {
        case <-time.After(time.Second):
            fmt.Println("hello")
        default:
            return
        }
    }
}

This has the beauty that you can also return an error (or any other value), and then it’s just:

// blocking
if err := foo(); err != nil {
    // do something with the err
}

3. Use tagged literals for struct initializations

This is a untagged literal example :

type T struct {
    Foo string
    Bar int
}

func main() {
    t := T{"example", 123} // untagged literal
    fmt.Printf("t %+vn", t)
}

Now if you go add a new field to your T struct your code will fail to compile:

type T struct {
    Foo string
    Bar int
    Qux string
}

func main() {
    t := T{"example", 123} // doesn't compile
    fmt.Printf("t %+vn", t)
}

Go’s compatibility rules (http://golang.org/doc/go1compat) covers your code if you use tagged literals. This was especially true when they introduced a new field called Zone to some net package types, see: http://golang.org/doc/go1.1#library. Now back to our example, always use tagged literals:

type T struct {
    Foo string
    Bar int
    Qux string
}

func main() {
    t := T{Foo: "example", Bar: 123}
    fmt.Printf("t %+vn", t)
}

This compiles fine and is scalable. It doesn’t matter if you add another field to the T struct. Your code will always compile and is guaranteed to be compiled by further Go versions. go vet will catch untagged struct literals, just run it on your codebase.

4. Split struct initializations into multiple lines

If you have more than 2 fields just use multiple lines. It makes your code much more easier to read, that means instead of:

T{Foo: "example", Bar:someLongVariable, Qux:anotherLongVariable, B: forgetToAddThisToo}

Use:

T{
    Foo: "example",
    Bar: someLongVariable,
    Qux: anotherLongVariable,
    B: forgetToAddThisToo,
}

This has several advantages, first it’s easier to read, second it makes disabling/enabling field initializations easy (just comment them out or removing), third adding another field is much more easier (adding a newline).

5. Add String() method for integers const values

If you are using custom integer types with iota for custom enums, always add a String() method. Let’s say you have this:

type State int

const (
    Running State = iota 
    Stopped
    Rebooting
    Terminated
)

If you create a new variable from this type and print it you’ll just get an integer (http://play.golang.org/p/V5VVFB05HB):

func main() {
    state := Running

    // print: "state 0"
    fmt.Println("state ", state)
}

Well here 0 doesn’t mean much until you lookup your consts variables again. Just adding the String() method to your State type fixes it (http://play.golang.org/p/ewMKl6K302):

func (s State) String() string {
    switch s {
    case Running:
        return "Running"
    case Stopped:
        return "Stopped"
    case Rebooting:
        return "Rebooting"
    case Terminated:
        return "Terminated"
    default:
        return "Unknown"
    }
}

The new output is: state: Running. As you see it’s now much more readable. It will make your life a lot of easier when you need to debug your app. You can do the same thing with by implementing the MarshalJSON(), UnmarshalJSON() methods etc..

As a final statement, this all can be automated now with the Stringer tool:

https://godoc.org/golang.org/x/tools/cmd/stringer

This tool uses go generate to create a very efficient String method automatically based on the integer type.

6. Start iota with a +1 increment

In our previous example we had something that is also open the bugs and I’ve encountered several times. Suppose you have a new struct type which also stores a State field:

type T struct {
    Name  string
    Port  int
    State State
}

Now if we create a new variable based on T and print it you’ll be surprised (http://play.golang.org/p/LPG2RF3y39) :

func main() {
    t := T{Name: "example", Port: 6666}

    // prints: "t {Name:example Port:6666 State:Running}"
    fmt.Printf("t %+vn", t)
}

Did you see the bug? Our State field is uninitialized and by default Go uses zero values of the respective type. Because State is an integer it’s going to be 0 and zero means basically Running in our case.

Now how do you know if the State is really initialized? Is it really in Running mode? There is no way to distinguish this one and is a way to cause unknown and unpredictable bugs. However fixing it easy, just start iota with a +1 offset (http://play.golang.org/p/VyAq-3OItv):

const (
    Running State = iota + 1
    Stopped
    Rebooting
    Terminated
)

Now your t variable will just print Unknown by default, neat right? :) :

func main() {
    t := T{Name: "example", Port: 6666}

    // prints: "t {Name:example Port:6666 State:Unknown}"
    fmt.Printf("t %+vn", t)
}

But starting your iota with a reasonable zero value is another way to solve this. For example you could just introduce a new state called Unknown and change it to:

const (
    Unknown State = iota 
    Running
    Stopped
    Rebooting
    Terminated
)

7. Return function calls

I’ve seen a lot of code like (http://play.golang.org/p/8Rz1EJwFTZ):

func bar() (string, error) {
    v, err := foo()
    if err != nil {
        return "", err
    }

    return v, nil
}

However you can just do:

func bar() (string, error) {
    return foo()
}

Simpler and easier to read (unless of course you want to log the intermediates values).

8. Convert slices,maps,etc.. into custom types

Converting slices or maps into custom types again and makes your code much easier to maintain. Suppose you have a Server type and a function that returns a list of servers:

type Server struct {
    Name string
}

func ListServers() []Server {
    return []Server{
        {Name: "Server1"},
        {Name: "Server2"},
        {Name: "Foo1"},
        {Name: "Foo2"},
    }
}

Now suppose you want to retrieve only servers that with a specific name. Let’s change our ListServers() function a little bit and add a simple filter support:

// ListServers returns a list of servers. If name is given only servers that
// contains the name is returned. An empty name returns all servers.
func ListServers(name string) []Server {
    servers := []Server{
        {Name: "Server1"},
        {Name: "Server2"},
        {Name: "Foo1"},
        {Name: "Foo2"},
    }

    // return all servers
    if name == "" {
        return servers
    }

    // return only filtered servers
    filtered := make([]Server, 0)

    for _, server := range servers {
        if strings.Contains(server.Name, name) {
            filtered = append(filtered, server)
        }
    }

    return filtered
}

Now you can use it for filter servers that has the Foo string:

func main() {
    servers := ListServers("Foo")

    // prints: "servers [{Name:Foo1} {Name:Foo2}]"
    fmt.Printf("servers %+vn", servers)
}

As you see our servers are now filtered. However this doesn’t scale well. What if you want to introduce another logic for your server set? Like checking health of all servers, creating a DB record for each server, filtering by another a new field, etc…

Let’s introduce another new type called Servers and change our initial ListServers() to return this new type:

type Servers []Server

// ListServers returns a list of servers.
func ListServers() Servers {
    return []Server{
        {Name: "Server1"},
        {Name: "Server2"},
        {Name: "Foo1"},
        {Name: "Foo2"},
    }
}

What we do now is, we just add a new Filter() method to our Servers type:

// Filter returns a list of servers that contains the given name. An
// empty name returns all servers.
func (s Servers) Filter(name string) Servers {
    filtered := make(Servers, 0)

    for _, server := range s {
        if strings.Contains(server.Name, name) {
            filtered = append(filtered, server)
        }

    }

    return filtered
}

And now let us filter servers with the Foo string:

func main() {
    servers := ListServers()
    servers = servers.Filter("Foo")
    fmt.Printf("servers %+vn", servers)
}

Voila! See how your code just simplified? You want to check if the servers are healthy? Or add a DB record for each of the server? No problem just add those new methods:

func (s Servers) Check() 
func (s Servers) AddRecord() 
func (s Servers) Len()
...

9. withContext wrapper functions

Sometimes you do repetitive stuff for every function, like locking/unlocking, initializing a new local context, preparing initial variables, etc.. An example would be:

func foo() {
    mu.Lock()
    defer mu.Unlock()

    // foo related stuff
}

func bar() {
    mu.Lock()
    defer mu.Unlock()

    // bar related stuff
}

func qux() {
    mu.Lock()
    defer mu.Unlock()

    // qux related stuff
}

If you want to change one thing, you need to go and change them all in other places. If its common task the best thing is to create a withContext function. This function takes a function as an argument and calls it with the given context:

func withLockContext(fn func()) {
    mu.Lock
    defer mu.Unlock()

    fn()
}

Then just refactor your initial functions to make use of this context wrapper:

func foo() {
    withLockContext(func() {
        // foo related stuff
    })
}

func bar() {
    withLockContext(func() {
        // bar related stuff
    })
}

func qux() {
    withLockContext(func() {
        // qux related stuff
    })
}

Don’t just think of a locking context. The best use case for this is a DB connection or a DB context. Let’s slightly change our withContext function:

func withDBContext(fn func(db DB)) error {
    // get a db connection from the connection pool
    dbConn := NewDB()

    return fn(dbConn)
}

As you see now it gets a connection, passes it to the given function and returns the error of the function call. Now all you do is:

func foo() {
    withDBContext(func(db *DB) error {
        // foo related stuff
    })
}

func bar() {
    withDBContext(func(db *DB) error {
        // bar related stuff
    })
}

func qux() {
    withDBContext(func(db *DB) error {
        // qux related stuff
    })
}

You changed to mind to use a different approach, like making some pre initialization stuff? No problem, just add them into withDBContext and you are good to go. This also works perfect for tests.

This approach has the disadvantage that it pushes out the indentation and makes it harder to read. Again seek always the simplest solution.

10. Add setter, getters for map access

If you are using maps heavily for retrieving and adding use always getters and setters around your map. By using getters and setters you can encapsulate the logic to their respective functions. The most common error made here is concurrent access. Say you have this in one goroutine:

m["foo"] = bar

And this on another:

delete(m, "foo")

What happens? Most of you are already familiar to race conditions like this. Basically this is a simple race condition because maps are not thread safe by default. But you can easily protect them with mutexes:

mu.Lock() m["foo"] = "bar" mu.Unlock()

And:

mu.Lock() delete(m, "foo") mu.Unlock()

Suppose you are using this map in other places. You need to go and put everywhere mutexes! However you can avoid this entirely by using getter and setter functions:

func Put(key, value string) {
    mu.Lock()
    m[key] = value
    mu.Unlock()
}




func Delete(key string) {
    mu.Lock()
    delete(m, key)
    mu.Unlock()
}

An improvement over this procedure would be using an interface. You could completely hide the implementation. Just use a simple, well defined interface and let the package users use them:

type Storage interface {
    Delete(key string)
    Get(key string) string
    Put(key, value string)
}

This is just an example but you get the idea. It doesn’t matter what you use for the underlying implementation. What matters is the usage itself and an interface simplifies and solves lots of the bugs you’ll encounter if you expose your internal data structures.

Having said that, sometimes an interface is just and overkill because you might have a need to lock several variables at once. Know you application well and apply this improvement only if you have a need for it.

Conclusion

Abstractions are not always good. Sometimes the most simplest thing is just the way you’re doing it already. Having said that, don’t try to make your code smarter. Go is by nature a simple language, in most cases it has only one way to do something. The power comes from this simplicity and it is one of the reasons why it’s scaling so well on the human level.

Use these techniques if you really need them. For example converting a []Server to Servers is another abstraction, do it only if you have a valid reason for it. But some of the techniques like starting iotas with 1 could be used always. Again always strike in favor of simplicity.

A special thanks to Cihangir Savas, Andrew Gerrand, Ben Johnson and Damian Gryski for their valuable feedback and suggestions.