Fix default arguments + add bugbear

[araffin]

Description

Fixed potential issue when calling off-policy algorithms with default arguments multiple times (the size of the replay buffer would be the same)
and add bug bear extension to test requirements.

Motivation and Context

• I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)

Checklist:

• I've read the CONTRIBUTION guide (required)
• I have updated the changelog accordingly (required).
• My change requires a change to the documentation.
• I have updated the tests accordingly (required for a bug fix or a new feature).
• I have updated the documentation accordingly.
• I have reformatted the code using make format (required)
• I have checked the codestyle using make check-codestyle and make lint (required)
• I have ensured make pytest and make type both pass. (required)
• I have checked that the documentation builds using make doc (required)

Note: You can run most of the checks using make commit-checks.

Note: we are using a maximum length of 127 characters per line

[Miffyli]

Bit of clarification: does this happen when off-policy algo is constructed, and in what kind of scenario? This is more of a question for the thread, but could also be updated here in changelog.

Edit: Other than that LGTM :). Some nice tweaks and cleaning up.

[araffin]

For a concrete example:

class A:
    def __init__(self, a=int(1e6)):
        self.a = a


obj = A()
obj2 = A()

print(id(obj.a), id(obj2.a))
# obj.a and obj2.a are pointing to the same object
assert id(obj.a) == id(obj2.a)

In our very special case, it is ok-ish because buffer_size is an int (so if you modify the value, it won't modify the value of the other object, but it is just a bad practice that can be very painful to debug.

[Miffyli]

Thanks for the clarification! LGTM :)