added configure_mappers call to alchemy scaffold

[ztane]

I propose the following fix to the SQLAlchemy ORM to add the explicit configure_mappers call - not having it makes the code prone to really hard-to-find heisenbugs that trigger only when no model object instances were created prior to accessing the backreffed properties and such.

The docs would be out of date (I can update them if this is deemed an acceptable change).

[mcdonc]

I don't know enough about SQLAlchemy to make a pronouncement on inclusion of this patch. @mmerickel?

[mmerickel]

Yeah I've been hit with this in the past. The best place to do this is in an init_model function but since the scaffold doesn't have one this'll be the best spot for now. It needs to be done after all the models have been imported/defined.

[digitalresistor]

Should we update the scaffold to use the no-globals way of setting up Pyramid that has been discussed on IRC? I personally prefer it, and have my own scaffold locally that does that along with a bunch of other setup that I personally care about.

[mmerickel]

We have a branch[1] for this that needs some help.

[1] https://github.com/Pylons/pyramid/tree/feature/alchemy-scaffold-update

[pauleveritt]

I'm about to make an offer for which I'm likely not serious. [wink] How would everyone feel about making Wichert's pyramid_sqlalchemy the recommended starting point?

If we did, we could move some of the best practices out of the scaffold and into pyramid_sqlalchemy, which is a better place. We could even move the scaffold out of Pyramid and into pyramid_sqlalchemy, making it easier to rev and lowering pain for Pyramid.

[pauleveritt]

Sorry, left out the offer...in July I could put some time into a branch in pyramid_sqlalchemy, as well as ask @mmerickel and @bertjwregeer and @ztane and perhaps others what best practices we're behind on. I could then update the quick tutorial/tour.

[mmerickel]

The last time I looked at pyramid_sqlalchemy it used several patterns that I really didn't like. It seems to serve the purpose of making globals where there should be no globals. I'm happy to take another look though and discuss the motivations being the changes in the feature/sqlalchemy-scaffold-update branch.

[digitalresistor]

I don't use pyramid_sqlalchemy, and I am not sure that I want to add yet another dependency if it could just be added to the scaffold.

[pauleveritt]

I kind of viewed it as, wouldn’t it be good if we could take the things learned and agreed upon and update it in software releases. Scaffolds, once generated, are inert. But doing it in a scaffold is still an improvement. I can pitch in by taking those decisions and updating the tutorials to reflect them.

Want to list the best practices as comments here, or should I just watch for activity on this branch?

—Paul

On Jun 18, 2015, at 9:21 PM, Bert JW Regeer [email protected] wrote:

I don't use pyramid_sqlalchemy, and I am not sure that I want to add yet another dependency if it could just be added to the scaffold.

—
Reply to this email directly or view it on GitHub #1657 (comment).

[blaflamme]

@pauleveritt basically what we have talked about is something similar to this https://gist.github.com/blaflamme/0b25c8fed8db0b87b9b6

[pauleveritt]

There were also some things discussed about namespacing, to let Alembic work seamlessly, IIRC.

Did I also see something about allowing models to be loaded lazily or something?

—Paul

On Jun 19, 2015, at 11:07 AM, Blaise Laflamme [email protected] wrote:

@pauleveritt https://github.com/pauleveritt basically what we have talked about is something similar to this https://gist.github.com/blaflamme/0b25c8fed8db0b87b9b6 https://gist.github.com/blaflamme/0b25c8fed8db0b87b9b6
—
Reply to this email directly or view it on GitHub #1657 (comment).

[blaflamme]

@pauleveritt it was only the pattern for being not global

[pauleveritt]

I have a question about getting at the session, under this approach, when there is no request and no config in scope. For example, I have the following method on a model class:

    def __getitem__(self, key):
        try:
            return Session.query(Node).filter_by(
                name=key, parent=self).one()
        except NoResultFound:
            raise KeyError(key)

...where "Session" came from pyramid_sqlalchemy. The new get_session function in this scaffold wants to be passed in the dbmaker function, which wants the engine, which wants the settings, which wants config. Any suggestions?

[mmerickel]

You'll need to pass the session down through your traversal hierarchy from parent to child.

Alternatively if you have a handle to a managed object then you can get the session from it using dbsession = sqlalchemy.orm.object_session(obj).

So in most apps you would probably be passing an orm object down through the __init__(self, obj) and from there you would just pull the session from the obj.

If your __getitem__ is defined on an orm object itself then obviously you can use object_session(self).

[pauleveritt]

On Sep 14, 2015, at 5:39 PM, Michael Merickel [email protected] wrote:

You'll need to pass the session down through your traversal hierarchy from parent to child.

Presumably that means I can’t use the default traverser, as its getitem protocol only provides the key.

Alternatively if you have a handle to a managed object then you can get the session from it using dbsession = sqlalchemy.orm.object_session(obj).

Yep, that worked, thanks.

—Paul

[mmerickel]

On Sep 14, 2015, at 5:39 PM, Michael Merickel [email protected] wrote:

You'll need to pass the session down through your traversal hierarchy from parent to child.
Presumably that means I can’t use the default traverser, as its getitem protocol only provides the key.

Well you normally control the creation of the children in the parent so you can pass it down. I know this isn't how you do it in ZODB but in a more manual hierarchy it's possible.

[pauleveritt]

On Sep 14, 2015, at 6:14 PM, Michael Merickel [email protected] wrote:

On Sep 14, 2015, at 5:39 PM, Michael Merickel [email protected] mailto:[email protected] wrote:

You'll need to pass the session down through your traversal hierarchy from parent to child.
Presumably that means I can’t use the default traverser, as its getitem protocol only provides the key.

Well you normally control the creation of the children in the parent so you can pass it down. I know this isn't how you do it in ZODB but in a more manual hierarchy it's possible.

One thing I’m investigating is a custom traverser combined with recursive CTEs to take a crack at eliminating the complaints about “traversal means a lot of queries.” As such, passing it down might be natural. At the moment, though, I’m in good shape with a @Property for session which uses what you suggested.

Tres, I’m trying to move away from going back to Python hopping around the graph, to eliminate queries.

—Paul

[ztane]

@pauleveritt I just yesterday did traversal on SQLAlchemy. It turns out you do not need session at all, just use lazy='dynamic' on the child link:

class Page(Base):
    __tablename__ = 'page'
    id = Column(Integer, primary_key=True)
    slug = Column(Text)
    parent_id = Column(Integer, ForeignKey('page.id'))

    children = relationship(
        "Page",
        lazy='dynamic',
        backref=backref('parent', remote_side=[id])
    )

    @property
    def __name__(self):
        return self.slug

    @property
    def __parent__(self):
        return self.parent

    def __getitem__(self, item: str):
        try:
            return self.children.filter_by(slug=item).one()

        except NoResultFound:
            raise KeyError(item)

[pauleveritt]

Great, thanks for that! I’ll update my code. (Though I still need the session for some other class methods.)

I also got some performance tips from recent commits in Kotti. I need to spend a little time on that as well.

—Paul

On Sep 24, 2015, at 8:56 AM, Antti Haapala [email protected] wrote:

@pauleveritt https://github.com/pauleveritt I just yesterday did traversal on SQLAlchemy. It turns out you do not need session at all, just use lazy='dynamic' on the child link:

class Page(Base):
tablename = 'page'
id = Column(Integer, primary_key=True)
slug = Column(Text)
parent_id = Column(Integer, ForeignKey('page.id'))

children = relationship(
    "Page",
    lazy='dynamic',
    backref=backref('parent', remote_side=[id])
)

@property
def __name__(self):
    return self.slug

@property
def __parent__(self):
    return self.parent

def __getitem__(self, item: str):
    try:
        return self.children.filter_by(slug=item).one()

    except NoResultFound:
        raise KeyError(item)

—
Reply to this email directly or view it on GitHub #1657 (comment).

[mmerickel]

We should apply these changes as part of #2024. The way that's done will change when it is no longer a single module of models so I'm closing this and adding a todo.

[ergo]

https://github.com/Pylons/pyramid/pull/2098/files
This should be fixed now in the scaffold branch.