Skip to content

Replace bulk Spanner mutation calls w/ plain SQL #369

Open
@pjenvey

Description

@pjenvey

batch commit's do_append now makes an efficient bulk insert via INSERT INTO .. SELECT FROM UNNEST(...) instead of using spanner's mutation API.

In my local testing this performed similarly to a mutation bulk insert (if anything the mutation was slightly faster, but pretty much negligible).

Avoiding mutations is preferable as they don't play nicely with our unit tests' transactions. We've worked around this for put_bso by implementing both mutation and individual INSERT versions, with the mutation cfg toggled off during tests.

put_bso could definitely switch to this INSERT instead of a mutation and it may be possible to implement an UPDATE utilizing UNNEST(...) in a similar fashion -- looking similar to batch_commit_update.sql but selecting from UNNEST(...) instead of batch_bsos -- and hopefully Spanner would optimize it efficiently.

If so we could remove the mutation impls entirely.

Metadata

Metadata

Assignees

No one assigned

    Labels

    5Estimate - l - Moderately complex, will require some effort but clearly defined.cleanup

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions