-
Notifications
You must be signed in to change notification settings - Fork 0
Process: pushing to staging and production (heroku)
We had a meeting to discuss our company's processes for pushing to staging and production. This document's top section describes the rake tasks that we'll write; full meeting notes that record why we made certain decisions are below.
Stacia, Jen-Mei, Judy, Curtis, Bonnie, Mason 11am Tuesday 11/27 in the classroom
- FETCH from origin (NOT a pull). compare origin/master with local/master. if local/master has diverged or is not an ancestor of origin/master, then abort. (basically, let it happen if they are exactly equal, or if local/master is an ancestor of origin/master. origin/master can be ahead ("my coworker just pushed this change that i don't want to push yet"); origin/master can NEVER be behind during a push to staging.) <--CI will take care of this later, but until we set up CI, we need the rake task to do this. (is this correct?)
- tag a commit in local/master
- push the tag to origin: git tag --push
- us the tag to push to staging. git push staging tag name:master that will push my local reference to this tag to staging. then do the heroku stuff: heroku rake db:migrate, db:seed
- task takes a mandatory argument: the tag to push (it is a staging tag)
- makes a new tag: production_today's datetime (in human readable format, with no special characters)
- push to origin
- use tag to push to production
We need to integrate Jenkins into this process. This won't happen today, but soon (especially for new projects).
We also need to template this so we can automate it.
- modify rake task for staging
- modify rake task for prod
- long-term set up CI
for Ananse: research ruby gems for setting up new projects
This meeting is to talk about how we have two different ways to push to production in our company. We've been using both on Goldilocks.
The two different styles that we will talk about now are:
- push from origin/master
- push from the last successful staging build.
Jen-Mei: sometimes, what is at staging/master is not necessarily on origin. That needs to not happen. Jen-Mei: we should never do a production push from origin/master. We should push from origin/staging, or a tag. Because what your'e doing is pushing from your local copy.
Could make sure they're equal by using a rake task by pushing to both
We're just naming the commit; the commit have several names. Tags are the only things that stick with that commit. The name of a BRANCH (origin/master, origin/staging) can change, because the pointer changes to the latest commit on that branch.
So pushing the tag is the right thing to do.
All agree on pushing a tag -- we already do this in either way.
People have been tagging, then pushing origin master -- that's wrong. you should push, then tag.
What we're doing is: we're pushing from your computer. So we're never actually pushing "from github" --that's an illusion.
Make sure that github and staging are the same.
Can we push origin to staging? well you can't push origin to staging directly, but you CAN push your own local copy of origin/master to staging. Do we tag before or after push? Answer: BEFORE, because if it doesn't work
To push to staging:
- FETCH from origin (NOT a pull). if local/master is ahead of origin/master then abort. (how to do this? git ancestor) <--curtis would have done a git log and grepped stuff. there exist ruby libraries that can do this. if local/master is not equal to or a descendant of origin/master, then abort. is there a reason not to just fetch from origin/master and merge and push all of it to staging? no, don't do this, because what if there's a weird conflict that your coworker just pushed to origin? if on jenkins, wouldn't have this problem. it would be working. the more we talk about this, the more we should do the jenkins thing.
- tag a commit in local/master
- push the tag to origin: git tag --push
- us the tag to push to staging. git push staging tag name:master that will push my local reference to this tag to staging. then do the heroku stuff: heroku rake db:migrate, db:seed
"master" is the default branch in the git repository. are we doing branching? jen-mei: people should make use of branches locally, but doesn't think we should maintain branches if we have to. if i'm working on something and i have to go, it makes sense to push my local branch to origin so that if i spill coffee on my computer, i don't lose my work. branching should NOT be a development process. it's a process that's local to developer. so as a team, we don't have a bunch of branches -- unless we have to (sometimes there are nontechnical reasons for this). so as practice, try to avoid branches unless it's for local purposes.
origin is the repo master is the name of the branch on origin a tag is a separate thing -- tags don't go on branches or anything. tags are just pointers to certain commits.
this is a lot of steps, but there will be rake tasks to automate this.
to be honest, we should be using jenkins. our automated tests should pass what people do with jenkins: every time you do a commit, it runs through CI. when it passes, it adds a tag showing it passes. when you say "push to staging," it looks for the most recent passed thing, and pushes THAT to staging. this is on all of our objective-c projects.
WE SHOULD INTEGRATE JENKINS INTO THIS PROCESS. We also need this on the next project too.
Talking about steps to push to production: Assuming that everything has been accepted: (If not everything has been accepted, then someone like the product champion has to decide whether it's acceptable to push that stuff or not. We're not talking about this process now. We're talking about the easy case, which is what w'ere aiming for.) Rake task: Sometimes, it is the case that the last staging tag doesn't work out. I would like the ability to specify an argument that is the tag I want. What would be easier: you MUST specify a tag; then if you don't specify a tag it says "you must specify a tag" So there's no default parameter. Won't run unless there's a tag.
** Do we like using rake for stuff like this? Jen-Mei brings up: there's also thor (a ruby thing that yehuda made) -- friendlier for things like parameters. rake is weird with parameters -- you have to put them in square brackets or something. Leave it up to Bonnie.
prepend production to the tag? don't need to; can look in tree to see which staging this production push is from. we talked about this a little. so we should make a new tag: production_[today's datetime] <--no colons in the time. be conscious that the datetime command will give you extra characters. you can give it a format.
- or you can give it an epoch (seconds since the beginning of time) … this was voted no on, because we like human-readable code better. so we'll use the date time, smushed together with no weird characters.
push to origin
use tag to push to production
What if my local
- FETCH from origin (NOT a pull). if local/master is ahead of origin/master then abort. (how to do this? git ancestor) <--curtis would have done a git log and grepped stuff. there exist ruby libraries that can do this. if local/master is not equal to or a descendant of origin/master, then abort. is there a reason not to just fetch from origin/master and merge and push all of it to staging? no, don't do this, because what if there's a weird conflict that your coworker just pushed ? if on jenkins, wouldn't have this problem. it would be working. the more we talk about this, the more we should do the jenkins thing.
And we could do what Mason said about automating the process to kick off a new project.
Question: do we need to do this for goldilocks today? no. we'll use the manual process (with modifications for reality). then we'll try this for ananse. then after it works for ananse, goldilocks will do it.
the point is: the "fetch from origin master, check if descendant" stuff won't' matter if we do it with jenkins. because we'll know that the tests work so we can push it. doesn't matter how far behind it is from master, because it'll test the new stuff. there's some change that the tests work but it doesn't work from a user's perspective. <--answer: you're pushing to staging first anyway, so someone should be checking it on staging. even with CI, MUST we
hotfixing without bypassing entire automated process? answer: have a hotfix rake task. we've had hotfixes on production every week. using cherry picking and building a new tag on top of the current production tag. jenmei would: create a branch off the last post to production; merge back to master (unless they can't, in which case there's a new pivotal tracker story to fix it)
mason brings up: hotfixing should not be so common that we have a process for it. curtis agrees. there's a situation where the client is putting on a lot of pressure saying "this change has to be on production immediately" -- this is something stacia can help us with. also goldilocks is a weird case because we ramped up like 12 people really quickly and now are ramping off.
what if we're getting ready to do acceptance and we shouldn't push to staging? we could have a freeze task. until it's unfrozen, you can't push to staging everyone should know "acceptance is happening right now, be cautious." it takes everyone being aware of how to toggle things on and off.
HOTFIXES ARE FOR EMERGENCIES, like "oh my god, anyone can log in with any password." we won't make a process for hotfixes. this is a client education issue.
we're going to do a git 2 tech talk which will cover some of this stuff.
what are we saying? if it's being accepted, please don't rewrite the site.
can jenkins handle this? jenmei needs to look at what the two mac minis have. paul and glenna are the two people that have been doing the most with these two mac minis. (jenkins and jenkins companion) mightyverse is on blazingci.
go over this once again: abort if local master has diverged or is not an ancestor of origin/master local master should never be ahead of origin/master
stacia: let's talk about whether this was appropriate to spend an hour on this. we said: yes. if we had this process, this would help us a lot as a group. we've never ahd a standard way to push to production, have ci, or have templates, so every time i work on a new project, i do the manual steps again (i don't bother writing the rake task because it's in my head), then we end up having to do the rake task eventually… we should make a list of everything to put in that template.
it'll get more complicated if we ever have to deploy on anything other than heroku. (like engine yard) fortunately we've been using mostly heroku lately.
we should make a heroku+ios app template. because this is common for us. the rails version changes so fast though. maybe it'll be bundle install. jen-mei and mason know about: there is something that people have created for kicking off new projects in rails, we don't know the names but search around for them. bonnie's todo: look around for these gems. it's a command that says "do this!" sets up gem file with stuff… may not be appropriate for us, but look into it.
(what if not rails? clojure on heroku! cat paws!)
I've also started to collect notes like these on a goldilocks-internal tumblr: Password is the typical blazing pair one.