Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add promotion code batch #1524

Merged
merged 11 commits into from
Jan 31, 2017

Conversation

vladstoick
Copy link
Contributor

@vladstoick vladstoick commented Oct 14, 2016

We (lostmy.name) have found that the current approach for creating promo codes doesn't work for creating thousands of promo codes as the request will time out by the time all codes are created. Therefore, we introduced promotion code batches to move the process of creating codes in a ActiveJob

This PR introduces a new model called Spree::PromotionCodeBatch which belongs to a Spree::Promotion. Each PromotionBatch has a base_code and a number_of_codes so it describes a batch of promo codes for a promotion.PromotionBatch also has an email field so we can send an email when all promo codes are created.

Generating the codes is done within a worker Spree::PromotionCodeBatchJob which calls Spree::PromotionCode::BatchBuilder. PromotionCode::BatchBuilder is very similar to Spree::PromotionCode::CodeBuilder) with a few optimisations (uses batches instead of creating all codes at once, uses more characters). BatchBuilder also updates PromotionCodeBatch's number_of_codes_processed so we can display a status in the view.

@vladstoick vladstoick changed the title Add promotion code batch [WIP] Add promotion code batch Oct 14, 2016
@vladstoick vladstoick force-pushed the promotion-code-batch branch 2 times, most recently from d7fa579 to 74b03de Compare October 14, 2016 12:13
@bbuchalter
Copy link
Contributor

Thanks for putting this forward. This is a regular pain point for our (tommyjohn.com) admins.

@vladstoick vladstoick force-pushed the promotion-code-batch branch from 74b03de to 8c9bef7 Compare October 14, 2016 15:45
@vladstoick vladstoick changed the title [WIP] Add promotion code batch Add promotion code batch Oct 14, 2016
@jordan-brough
Copy link
Contributor

jordan-brough commented Oct 14, 2016

This will be a fantastic improvement for shops that make lots of codes. 👍

Should we add a spree_promotion_codes.promotion_code_batch_id field? E.g. then admins could download all the codes for a particular batch.

end

codes_for_current_batch.map do |code|
promotion.codes.build(value: code)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we perhaps use Spree::PromotionCode.create! here instead of promotion.codes.build ... promotion.save!? That would avoid loading a million objects into memory if you're generating a million codes.

Copy link
Contributor

@jordan-brough jordan-brough Oct 14, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are breaking this up into batch_size, but since we're putting the codes on the promotion they'll all stick around in memory right? Maybe another solution could be to load our own promotion object after every batch_size iteration? (or reload the existing one)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's true. Doing a reload after save! should be enough right?


promotion.save!

promotion_code_batch.update!(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we add a spree_promotion_codes.promotion_code_batch_id column then we could eliminate this yes?

end

def sample_characters
@sample_characters ||= ('a'..'z').to_a + (2..9).to_a.map(&:to_s)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we make this a class_attribute as well, in case people want to customize what characters are used for codes?

@jordan-brough jordan-brough mentioned this pull request Oct 14, 2016
3 tasks
)
@promotion = @promotion_builder.promotion
@promotion = Spree::Promotion.new(permitted_resource_params)
@promotion.codes.new(value: params[:single_code]) if params[:single_code].present?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like how this separates creating a single code (via this controller) from creating a batch of codes (via the batch controller). There's probably even further improvement we can do along the lines of creating single codes, but it seems like that could be a separate PR. Possibly part of #1509?

</tbody>
</table>
<% else %>
<div class="alpha twelve columns no-objects-found">
Copy link
Contributor

@jhawthorn jhawthorn Oct 14, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We aren't using the skeleton classes anymore. This can juts be

<div class="no-objects-found">
  ...
</div>

@vladstoick vladstoick force-pushed the promotion-code-batch branch from 8c9bef7 to 683436e Compare October 14, 2016 21:21
<% admin_breadcrumb(plural_resource_name(Spree::PromotionCodeBatch)) %>

<%= form_tag admin_promotion_promotion_code_batches_path do %>
<%= fields_for :promotion_code_batch do |f| %>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This two lines can probably be combined into just a form_for

<% end %>
<div class="field" id="promotion_single_code_field">
<%= label_tag :single_code %>
<%= text_field_tag :single_code, @promotion.codes.first.try(:value), :readonly => [email protected]_record?, :class => 'fullwidth' %>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use try! and not try.

end
end

def build
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer a name other than build, which will be confusing since ActiveRecord uses that name a lot.

Any of build_codes, create_codes, process would be fine.

@vladstoick vladstoick force-pushed the promotion-code-batch branch 5 times, most recently from 9c4d234 to 85f3a98 Compare October 17, 2016 11:14
@vladstoick
Copy link
Contributor Author

@jordan-brough I've added a spree_promotion_codes.promotion_code_batch_id field and also a download button so you can download codes for a single batch

validates_presence_of :base_code, :number_of_codes

def status
if error.present?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you return some kind of status here instead? something with a symbol or something. so we can use locales to display the actual text. When we hardcode english like this, people that speak weird languages become sad and complain to me in private messages.

@@ -13,6 +13,9 @@
resources :promotion_rules
resources :promotion_actions
resources :promotion_codes, only: [:index]
resources :promotion_code_batches, only: [:index, :new, :create] do
get '/download', to: "promotion_code_batches#download"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we limit this to just respond to the :csv format?

def promotion_code_batch_finished(promotion_code_batch)
mail(
to: promotion_code_batch.email,
body: "All #{promotion_code_batch.number_of_codes} codes are created now",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you push these to locales too please


promotion.save!

promotion.reload
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whats the reason this is getting reloaded here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@jordan-brough jordan-brough Oct 20, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might want to leave a code comment to that effect? (about why the reload is needed)

it "sends an email email" do
expect(Spree::PromotionCodeBatchMailer)
.to have_received(:promotion_code_batch_errored)
expect(delivery).to have_received(:deliver_now)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we cut down the stubbing in the mailers and instead look at the test deliveries to see if they contain the content we're expecting?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A great start in email testing are the Rails guides http://guides.rubyonrails.org/testing.html#testing-your-mailers

end

it "builds codes with distinct values" do
expect(subject.promotion.codes.map(&:value).uniq.size).to eq(number_of_codes)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you hardcode number_of_codes for all these tests? its more clear to asset something is equal to a specific thing rather than a variable.

context "with a batch that already started" do
before do
allow(subject).to receive_message_chain("promotion_codes.count")
.and_return(1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you kill the stubbing in this? I don't think its too costly to create a promo batch with a code to avoid the message chain


expect(Spree::PromotionCodeBatchJob)
.to have_received(:perform_later)
.with(subject)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this the same thing that was stubbed above? is there a way to kill those two things?

it "sends a success email" do
expect(Spree::PromotionCodeBatchMailer)
.to have_received(:promotion_code_batch_finished)
expect(delivery).to have_received(:deliver_now)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please expect ActionMailer::Base.deliveries to be present instead of stubbing implementation details. Thanks

it "sends an email email" do
expect(Spree::PromotionCodeBatchMailer)
.to have_received(:promotion_code_batch_errored)
expect(delivery).to have_received(:deliver_now)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A great start in email testing are the Rails guides http://guides.rubyonrails.org/testing.html#testing-your-mailers

<table>
<thead>
<tr>
<th>Base code</th>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't hardcore English in table headers, always use the model attributes translations.

<td><%= promotion_code_batch.base_code %></td>
<td><%= promotion_code_batch.number_of_codes %></td>
<td>
<%= promotion_code_batch.status %>.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As Clarke already mentioned, please return a symbol as status and pass the status here to Spree.t with a meaningful scope (ie. promotion_code_batches.status, so this can be translated in foreign languages.

@vladstoick vladstoick force-pushed the promotion-code-batch branch 2 times, most recently from 7d64694 to ff5bd7d Compare October 19, 2016 14:43
@vladstoick
Copy link
Contributor Author

vladstoick commented Oct 19, 2016

@cbrunsdon, @tvdeyen: I think I went over everything you mentioned

  • Use #.t everywhere + updated en.yml file
  • Fix mail specs: I am now using email = ActionMailer::Base.deliveries.last to check that the email has the correct details.
  • download route now has a default format of csv
  • Fixed various where too many things were specs stubbed
  • I've moved the #status method in the view as it felt that even if we were returning a :symbol we were still doing all those checks in the view to use the correct translations. I can move it back / to a helper if the logic seems too complicated for the view

Can you have a look over this again when you have time?

delegate :promotion, :number_of_codes, :base_code, to: :promotion_code_batch

class_attribute :random_code_length, :batch_size, :sample_characters
self.random_code_length = 8
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer we kept the old value here 6, just to avoid surprise changes to users.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done :)

@vladstoick vladstoick force-pushed the promotion-code-batch branch 3 times, most recently from 1b652fe to afa55b6 Compare October 20, 2016 08:39
@vladstoick vladstoick force-pushed the promotion-code-batch branch from a2f392b to a3babeb Compare October 23, 2016 10:16
@vladstoick
Copy link
Contributor Author

@tvdeyen, @cbrunsdon did you have a chance to look over this again?

Copy link
Member

@tvdeyen tvdeyen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Altogether great work! Really like the feature and the implementation 👍

Have some minor nits about translation scopes, though.

format.csv do
filename = "promotion-code-batch-list-#{@promotion_code_batch.id}.csv"
headers["Content-Type"] = "text/csv"
headers["Content-disposition"] = "attachment; filename=\"#{filename}\""
Copy link
Member

@tvdeyen tvdeyen Nov 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use Rails' send_file here? Then you can spare to set headers, Rails does this for you.

<table>
<thead>
<tr>
<th><%= Spree.t("promotion_code_batches.base_code") %></th>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't invent your own translation keys, if Rails already has one for you.
Use Spree::PromotionCodeBatch.human_attribute_name(:COLUMN_NAME) instead.

Thanks

mail(
to: promotion_code_batch.email,
body: Spree.t(
"promotion_code_batches.errored",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrong indention

@@ -0,0 +1,6 @@
CSV.generate do |csv|
csv << ['Code']
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normally this header should be translatable, but this view could also be overwritten in the host app. So, I'm not super keen about it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if I agree with CSV headers being i18n'd, as they might need to be parsed by external systems... so I'm also a toss up. So I'm probably good with this as a default, and as Thomas says letting people override if they don't like.

<%= bf.text_field :number_of_codes, :readonly => [email protected]_record?, class: 'fullwidth' %>
<% end %>
<% end %>
<div class="field" id="promotion_single_code_field">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we don't use this ID anywhere we should rewrite this into a data-hook attribute, if I assume correctly that you added this for all the defacers out there.

number_of_codes: promotion_code_batch.number_of_codes
),
subject: Spree.t(
"promotion_code_batches.email_subject.batch_finished",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rails already has i18n scopes for mailer subjects. This mailer has the scope:

en:
  spree:
    promotion_batch_mailer:
      promotion_code_batch_finished:
        subject: "Your batches are ready!"

def promotion_code_batch_finished(promotion_code_batch)
mail(
to: promotion_code_batch.email,
body: Spree.t(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like to hard code the body in here. Please let's use mailer views as for all the other mailers.

def promotion_code_batch_errored(promotion_code_batch)
mail(
to: promotion_code_batch.email,
body: Spree.t(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here: We shouldn't hard code the body in here.

"promotion_code_batches.errored",
error: promotion_code_batch.error
),
subject: Spree.t(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above: We don't need to introduce our own i18n scope.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, you are passing the id in here. That's why you need you own translation scope.

Hmm, I doubt that any user has a real understanding where this id is coming from and what to do with it.

Maybe a link to the failed batch in the admin would be much more useful. And this correlates to my former comment: Let's use mailer views instead of a hard coded body.

sample_characters.sample
end.join

"#{base_code}_#{suffix}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This concatenation can not be configured anywhere, right? We should make this configurable, as I already tickets in my backlog ("remove the underscore form promotion codes") 😆

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I say ignore thomas and let him work on his own backlog, as this PR is separate from that feature.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this was just moved from old implementation, I'm fine with keeping it. 👍

Copy link
Contributor

@cbrunsdon cbrunsdon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking really good, almost there! Mostly I just agree with all the of Thomas' i18n changes.

@@ -0,0 +1,6 @@
CSV.generate do |csv|
csv << ['Code']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if I agree with CSV headers being i18n'd, as they might need to be parsed by external systems... so I'm also a toss up. So I'm probably good with this as a default, and as Thomas says letting people override if they don't like.

sample_characters.sample
end.join

"#{base_code}_#{suffix}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I say ignore thomas and let him work on his own backlog, as this PR is separate from that feature.

if promotion_codes.count.zero?
PromotionCodeBatchJob.perform_later(self)
else
raise "Batch already started"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you create a CantProcessStartedBatch exception and call that instead of the raise please? Makes finding different exceptions in external systems way easier.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 great catch @cbrunsdon

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also won't prevent process from being called multiple times, since no codes will be created until the job starts running. Worth considering if we care about this race condition.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jhawthorn good point. Since this model represents a "batch" which can't reasonably be completed within a transaction, perhaps we should have some sort of state column on it and not just infer the state by counting the codes on it. Maybe started_at/errored_at/finished_at columns? Or a state column?

.promotion_code_batch_finished(promotion_code_batch)
.deliver_now
rescue => e
promotion_code_batch.update!(error: e.inspect)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless there's a kind of error we're expecting I don't see a reason to store the error on the model.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jhawthorn this will make the admin UI more helpful. If a batch fails you can go in and see the reason that it fails. E.g. a customer service rep tries to create a promo, it fails for some reason, they tell a dev, and the dev can easily see what happened (and go look in their exception reporting tool or whatever for more details).

Spree::PromotionCodeBatchMailer
.promotion_code_batch_finished(promotion_code_batch)
.deliver_now
rescue => e
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer the rescuing and mailing was done in the PromotionCodeBatchJob. IMO the reason we want to send an email and set the error on the model is that this is being run in the background.

Moving that to the job class reduces the responsibilities of this class and should make it better to test, and more flexible. Maybe a developer wants to generate promotion codes without sending emails.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving the email to the Job makes sense to me but I think setting the error on the promo_code_batch should stay here. We allow a row to exist in the spree_promotion_code_batches table when it may have failed part way through, so marking it with an error makes sense to me, and this seems like the place to do it.

column: :promotion_id
)

add_column(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add an index on this column

class CreateSpreePromotionCodeBatch < ActiveRecord::Migration[5.0]
def change
create_table :spree_promotion_code_batches do |t|
t.references :promotion, null: false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add index: true on this

@vladstoick
Copy link
Contributor Author

vladstoick commented Nov 28, 2016

I am adding a comment here as some changes are now outdated
@tvdeyen:

Regarding: send_file:

I just used what was used in promotion_codes_controller. However, is it possible to use send_file for this? I was able to use send_data and render_to_string.

Email body & URLs:

I have moved the body to individual erb files but I have noticed there's no default action_mailer.default_url_options so I can't generate URLs (I could add that in the spec but then by default the sandbox would have errors). I have added the promotion name in the meanwhile.

@cbrunsdon, @jordan-brough I think I covered all of your concerns. I have added a new column state

I have left all my commit history - easier to see changes this way

@vladstoick
Copy link
Contributor Author

@cbrunsdon, @tvdeyen, @jordan-brough: Is this still being consider or should I close this PR?

@cbrunsdon
Copy link
Contributor

Hey, sorry @vladstoick, I am super interested in getting this merged into core I just lost sight of the PR and you have my apologies, thanks for sticking through and poking us again.

Copy link
Member

@tvdeyen tvdeyen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks and sorry for the delay. Great work! 👌🏻

@jordan-brough jordan-brough merged commit 0d2f986 into solidusio:master Jan 31, 2017
@vladstoick vladstoick deleted the promotion-code-batch branch January 31, 2017 20:25
@vladstoick vladstoick restored the promotion-code-batch branch January 31, 2017 20:25
@vladstoick vladstoick deleted the promotion-code-batch branch February 6, 2017 11:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants