Compare commits

..

No commits in common. "devel" and "1.2.0" have entirely different histories.
devel ... 1.2.0

19 changed files with 2075 additions and 2632 deletions

View File

@ -8,7 +8,7 @@
set -e;
CI_CACHE=$HOME/.cache;
INSTALL_POETRY_VERSION="${POETRY_VERSION:-1.4.1}";
POETRY_VERSION=1.2.2;
mkdir --parents "$CI_CACHE";
@ -26,10 +26,9 @@ poetry --version --no-ansi;
poetry run pip --version;
poetry install \
--sync \
--no-ansi \
--no-root \
--only ci;
--quiet \
--remove-untracked \
--no-ansi;
poetry env info;
poetry run tox --version;

View File

@ -5,11 +5,9 @@ on:
types: ["opened", "synchronize"]
push:
branches: ["devel"]
env:
POETRY_VERSION: 1.4.1
jobs:
Test:
name: Python ${{ matrix.python.version }}
name: Test with Python ${{ matrix.python.version }}
runs-on: ubuntu-latest
strategy:
matrix:
@ -22,8 +20,6 @@ jobs:
toxenv: py39
- version: "3.10"
toxenv: py310
- version: "3.11"
toxenv: py311
fail-fast: true
steps:
- name: Checkout
@ -55,6 +51,7 @@ jobs:
- name: Run Toxenv ${{ matrix.python.toxenv }}
run: poetry run tox -e ${{ matrix.python.toxenv }}
Check:
name: Security, Linting, Formatting, Typing
runs-on: ubuntu-latest
steps:
- name: Checkout

View File

@ -11,6 +11,7 @@
# --disable=W"
disable=logging-fstring-interpolation
,logging-format-interpolation
,bad-continuation
,line-too-long
,ungrouped-imports
,typecheck

View File

@ -2,24 +2,6 @@
See also: [Github Release Page](https://github.com/enpaul/peewee-plus/releases).
## Version 1.3.0
View this release on: [Github](https://github.com/enpaul/peewee-plus/releases/tag/1.3.0),
[PyPI](https://pypi.org/project/peewee-plus/1.3.0/)
- Add field for storing Datetime Timedeltas
- **BREAKING CHANGE**: Update `JSONField` to raise `ValueError` instead of
`IntegrityError` on bad data write
- Update tests to account for more cases and test public API validity
## Version 1.2.1
View this release on: [Github](https://github.com/enpaul/peewee-plus/releases/tag/1.2.1),
[PyPI](https://pypi.org/project/peewee-plus/1.2.1/)
- Add PyPI classifier for Python 3.11
- Fix SQLite variable limit determination to account for changes in SQLite 3.32
## Version 1.2.0
View this release on: [Github](https://github.com/enpaul/peewee-plus/releases/tag/1.2.0),

View File

@ -27,10 +27,9 @@ Examples of unacceptable behavior include:
- The use of sexualized language or imagery, and sexual attention or advances of any kind
- Trolling, insulting or derogatory comments, and personal or political attacks
- Public or private harassment
- Publishing others' private information, such as a physical or email address, without
their explicit permission
- Other conduct which could reasonably be considered inappropriate in a professional
setting
- Publishing others' private information, such as a physical or email address, without their
explicit permission
- Other conduct which could reasonably be considered inappropriate in a professional setting
## Enforcement Responsibilities
@ -53,8 +52,8 @@ offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the
community leaders responsible for enforcement at \[INSERT CONTACT METHOD\]. All complaints
will be reviewed and investigated promptly and fairly.
community leaders responsible for enforcement at \[INSERT CONTACT METHOD\]. All
complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the reporter of
any incident.
@ -106,8 +105,8 @@ toward or disparagement of classes of individuals.
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 2.0,
available at https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
Community Impact Guidelines were inspired by
[Mozilla's code of conduct enforcement ladder](https://github.com/mozilla/diversity).
Community Impact Guidelines were inspired by [Mozilla's code of conduct
enforcement ladder](https://github.com/mozilla/diversity).
For answers to common questions about this code of conduct, see the FAQ at
https://www.contributor-covenant.org/faq. Translations are available at

View File

@ -39,5 +39,5 @@ publish: clean test build ## Build and upload to pypi (requires $PYPI_API_KEY be
@poetry publish --username __token__ --password $(PYPI_API_KEY)
dev: ## Create local dev environment
poetry install --sync --with dev --with ci --with test --with security --with static
poetry install --sync
poetry run pre-commit install

View File

@ -14,8 +14,7 @@ release history.
## Documentation
*The documentation for this project is currently a work in progress. Please see the source
code for complete docs*
*The documentation for this project is currently a work in progress. Please see the source code for complete docs*
- [Installing](#installing)
- [Features](#features)
@ -23,7 +22,7 @@ code for complete docs*
## Installing
peewee+ is [available on PyPI](https://pypi.org/project/peewee-plus/) and can be installed
Peewee+ is [available on PyPI](https://pypi.org/project/peewee-plus/) and can be installed
using Poetry, Pipenv, or Pip:
```bash
@ -58,36 +57,35 @@ when using SQLite
### Functions
`calc_batch_size` - Helper function for writing backend-agnostic batch queries while
accounting for the
[SQLite max variable limit](https://www.sqlite.org/limits.html#max_variable_number).
[`calc_batch_size`](https://github.com/enpaul/peewee-plus/blob/1.0.0/peewee_plus.py#L71) -
Helper function for determining how to batch a create/update query with SQLite
`flat_transaction` - Decorator function for wrapping callables in a database transaction
without creating nested transactions
[`flat_transaction`](https://github.com/enpaul/peewee-plus/blob/devel/peewee_plus.py#L137)
\- Decorator function for wrapping callables in a database transaction without creating
nested transactions
### Classes
`PathField` - A Peewee database field for storing
[`PathField`](https://github.com/enpaul/peewee-plus/blob/1.0.0/peewee_plus.py#179) - A
Peewee database field for storing
[Pathlib](https://docs.python.org/3/library/pathlib.html) objects, optionally relative to
a runtime value.
`PrecisionFloatField` - A Peewee database field for storing floats while specifying the
[`PrecisionFloatField`](https://github.com/enpaul/peewee-plus/blob/1.0.0/peewee_plus.py#L237)
\- A Peewee database field for storing floats while specifying the
[MySQL precision parameters](https://dev.mysql.com/doc/refman/8.0/en/floating-point-types.html)
`M` and `D`
`JSONField` - A Peewee database field for storing arbitrary JSON-serializable data
[`JSONField`](https://github.com/enpaul/peewee-plus/blob/1.0.0/peewee_plus.py#L267) - A
Peewee database field for storing arbitrary JSON-serializable data
`EnumField` - A Peewee database field for storing Enums by name
`TimedeltaField` A Peewee database field for natively storing
[`datetime.timedelta`](https://docs.python.org/3/library/datetime.html#datetime.timedelta)
objects
[`EnumField`](https://github.com/enpaul/peewee-plus/blob/1.0.0/peewee_plus.py#L322) - A
Peewee database field for storing Enums by name
## For Developers
All project contributors and participants are expected to adhere to the
[Contributor Covenant Code of Conduct, v2](CODE_OF_CONDUCT.md)
([external link](https://www.contributor-covenant.org/version/2/0/code_of_conduct/)).
[Contributor Covenant Code of Conduct, v2](CODE_OF_CONDUCT.md) ([external link](https://www.contributor-covenant.org/version/2/0/code_of_conduct/)).
The `devel` branch has the latest (and potentially unstable) changes. The stable releases
are tracked on [Github](https://github.com/enpaul/peewee-plus/releases),
@ -102,8 +100,8 @@ are tracked on [Github](https://github.com/enpaul/peewee-plus/releases),
[fork the repository](https://docs.github.com/en/enterprise/2.20/user/github/getting-started-with-github/fork-a-repo)
and [open a pull request](https://github.com/enpaul/peewee-plus/compare).
Developing this project requires [Python 3.10](https://www.python.org/downloads/) or later
and [Poetry 1.2](https://python-poetry.org/docs/#installation) or later. GNU Make can
Developing this project requires at least [Python 3.7](https://www.python.org/downloads/)
and at least [Poetry 1.0](https://python-poetry.org/docs/#installation). GNU Make can
optionally be used to quickly setup a local development environment, but this is not
required.

View File

@ -1,6 +1,4 @@
"""peewee+
Various extensions, helpers, and utilities for `Peewee`_
"""Peewee+
:constant SQLITE_DEFAULT_VARIABLE_LIMIT: The default number of variables that a single SQL query
can contain when interfacing with SQLite. The actual
@ -12,12 +10,9 @@ Various extensions, helpers, and utilities for `Peewee`_
SQLite database connection. The value for this constant is taken
directly from the `Peewee documentation`_
.. _`Peewee`: https://docs.peewee-orm.com/en/latest/
.. _`Peewee documentation`: https://docs.peewee-orm.com/en/latest/peewee/database.html#recommended-settings
.. _`Peewee documentation`: http://docs.peewee-orm.com/en/latest/peewee/database.html#recommended-settings
"""
import contextlib
import datetime
import enum
import functools
import json
@ -33,7 +28,7 @@ import peewee
__title__ = "peewee-plus"
__version__ = "1.3.0"
__version__ = "1.2.0"
__license__ = "MIT"
__summary__ = "Various extensions, helpers, and utilities for Peewee"
__url__ = "https://github.com/enpaul/peewee-plus/"
@ -55,7 +50,6 @@ __all__ = [
"PrecisionFloatField",
"SQLITE_DEFAULT_PRAGMAS",
"SQLITE_DEFAULT_VARIABLE_LIMIT",
"TimedeltaField",
]
@ -68,25 +62,7 @@ SQLITE_DEFAULT_PRAGMAS: Dict[str, Any] = {
}
SQLITE_DEFAULT_VARIABLE_LIMIT: int
# With SQLite 3.32 (2020-05-22) the devs bumped the default variable limit to
# 32766. This logic attemps to import the sqlite3 bindings and determine whether
# the version of the installed SQLite version is greater or equal to 3.32. If
# the sqlite3 bindings cannot be imported (either because they aren't installed)
# or because the platform is using SQLite 1 or 2 then it falls back to the
# 999 value.
try:
import sqlite3
except ImportError:
SQLITE_DEFAULT_VARIABLE_LIMIT = 999
else:
if sqlite3.sqlite_version_info[0] >= 3 or (
sqlite3.sqlite_version_info[0] == 3 and sqlite3.sqlite_version_info[1] >= 32
):
SQLITE_DEFAULT_VARIABLE_LIMIT = 32766
else:
SQLITE_DEFAULT_VARIABLE_LIMIT = 999
SQLITE_DEFAULT_VARIABLE_LIMIT: int = 999
T = TypeVar("T", bound=peewee.Model)
@ -278,9 +254,8 @@ class PrecisionFloatField(peewee.FloatField): # pylint: disable=abstract-method
.. _here: https://stackoverflow.com/a/67476045/5361209
:param max_digits: Maximum number of digits, combined from left and right of the decimal place,
to store for the value; corresponds to the ``M`` MySQL precision parameter.
:param decimal_places: Maximum number of digits that will be stored after the decimal place;
corresponds to the ``D`` MySQL precision parameter.
to store for the value.
:param decimal_places: Maximum number of digits that will be stored after the decimal place
"""
def __init__(self, *args, max_digits: int = 10, decimal_places: int = 4, **kwargs):
@ -295,7 +270,7 @@ class PrecisionFloatField(peewee.FloatField): # pylint: disable=abstract-method
class JSONField(peewee.TextField): # pylint: disable=abstract-method
"""Field class for storing JSON-serializable data
This field can be used to store a dictionary of data directly in the database
This field can be used to store a dictionary of data directly in the database without needing
without needing to call :func:`json.dumps` and :func:`json.loads` directly.
::
@ -317,8 +292,6 @@ class JSONField(peewee.TextField): # pylint: disable=abstract-method
:param dump_params: Additional keyword arguments to unpack into :func:`json.dump`
:param load_params: Additional keyword arguments to unpack into :func:`json.load`
:raises ValueError: When attempting to set a non-JSON serializable object to the field
:raises peewee.IntegrityError: When the underlying database value is not JSON serializable
"""
def __init__(
@ -336,7 +309,7 @@ class JSONField(peewee.TextField): # pylint: disable=abstract-method
try:
return super().db_value(json.dumps(value, **self.dump_params))
except TypeError as err:
raise ValueError(
raise peewee.IntegrityError(
f"Failed to JSON encode object of type '{type(value)}'"
) from err
@ -403,22 +376,3 @@ class EnumField(peewee.CharField): # pylint: disable=abstract-method
raise peewee.IntegrityError(
f"Enum {self.enumeration.__name__} has no value with name '{value}'"
) from None
class TimedeltaField(peewee.BigIntegerField):
"""Field class for storing python-native Timedelta objects
This is really just a helper wrapper around an integer field that performs the second conversions
automatically. It is a helpful helper though, so it's included.
.. note:: To avoid issues with float precision, this field stores the database value as an integer.
However, this necessitates the usage of the BigInt type to avoid overflowing the value.
Essentially, the value this field ends up storing is the number of microseconds in the
timedelta.
"""
def db_value(self, value: datetime.timedelta) -> int:
return super().db_value(int(value.total_seconds() * 1000000))
def python_value(self, value: int) -> datetime.timedelta:
return datetime.timedelta(seconds=super().python_value(value) / 1000000)

4268
poetry.lock generated

File diff suppressed because it is too large Load Diff

View File

@ -1,6 +1,6 @@
[tool.poetry]
name = "peewee-plus"
version = "1.3.0"
version = "1.2.0"
description = "Various extensions, helpers, and utilities for Peewee"
authors = ["Ethan Paul <24588726+enpaul@users.noreply.github.com>"]
repository = "https://github.com/enpaul/peewee-plus/"
@ -21,7 +21,6 @@ classifiers = [
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: Implementation :: CPython",
"Topic :: Database",
"Typing :: Typed"
@ -31,42 +30,25 @@ classifiers = [
python = "^3.7.1"
peewee = "^3.14.8"
[tool.poetry.group.test.dependencies]
[tool.poetry.dev-dependencies]
bandit = "^1.7.1"
black = "^22.8.0"
blacken-docs = "^1.12.0"
ipython = "^7.29.0"
mdformat = "^0.6.4"
mdformat-gfm = "^0.2"
mypy = "^0.910"
pre-commit = "^2.15.0"
pre-commit-hooks = "^4.0.1"
pylint = "^2.13.0"
pytest = "^6.2.5"
pytest-cov = "^3.0.0"
reorder-python-imports = "^2.6.0"
safety = "^2.2.0"
toml = "^0.10.2"
ruamel-yaml = {version = "^0.17.21", python = "^3.10"}
# This is a workaround for this issue with the Poetry export
# plugin which was blocking the 'security' CI check:
#
# https://github.com/python-poetry/poetry-plugin-export/issues/176
virtualenv = ">=20.15,<20.16"
[tool.poetry.group.ci.dependencies]
poetry = "^1.4.2"
tox = "^3.24.4"
tox-poetry-installer = {version = "^0.10.0", extras = ["poetry"]}
[tool.poetry.group.security.dependencies]
bandit = {version = "^1.7.1", python = "^3.10"}
poetry = {version = "^1.4.2", python = "^3.10"}
safety = {version = "^2.2.0", python = "^3.10"}
[tool.poetry.group.static.dependencies]
black = {version = "^22.8.0", python = "^3.10"}
blacken-docs = {version = "^1.12.0", python = "^3.10"}
mdformat = {version = "^0.7.16", python = "^3.10"}
mdformat-gfm = {version = "^0.3.5", python = "^3.10"}
mypy = {version = "^1.2.0", python = "^3.10"}
pre-commit = {version = "^2.15.0", python = "^3.10"}
pre-commit-hooks = {version = "^4.0.1", python = "^3.10"}
pylint = {version = "^2.13.0", python = "^3.10"}
reorder-python-imports = {version = "^2.6.0", python = "^3.10"}
toml = {version = "^0.10.2", python = "^3.10"}
types-toml = {version = "^0.10.1", python = "^3.10"}
[tool.poetry.group.dev.dependencies]
ipython = {version = "^8.10.0", python = "^3.10"}
tox-poetry-installer = {extras = ["poetry"], version = "^0.10.0"}
types-toml = "^0.10.1"
[build-system]
requires = ["poetry-core>=1.1.0"]

View File

@ -31,10 +31,3 @@ def test_about():
)
is True
)
def test_all():
"""Test that the string entries in ``__all__`` are correct"""
for item in peewee_plus.__all__:
assert hasattr(peewee_plus, item)

View File

@ -8,15 +8,6 @@ import peewee_plus
from .fixtures import fakedb
def test_public_api():
"""Test that the public API components are exposed via ``__all__``"""
assert peewee_plus.calc_batch_size.__name__ in peewee_plus.__all__
assert peewee_plus.flat_transaction.__name__ in peewee_plus.__all__
assert "SQLITE_DEFAULT_VARIABLE_LIMIT" in peewee_plus.__all__
assert "SQLITE_DEFAULT_PRAGMAS" in peewee_plus.__all__
def test_sqlite(fakedb):
"""Test the calculation of batch sizes on SQLite"""
@ -26,11 +17,7 @@ def test_sqlite(fakedb):
data = peewee.IntegerField()
# Three is just chosen as an arbitrary multiplier to ensure the value is larger than the
# sqlite variable limit
models = [
TestModel(item) for item in range(peewee_plus.SQLITE_DEFAULT_VARIABLE_LIMIT * 3)
]
models = [TestModel(item) for item in range(500)]
assert (
peewee_plus.calc_batch_size(models) <= peewee_plus.SQLITE_DEFAULT_VARIABLE_LIMIT
)

View File

@ -11,12 +11,6 @@ import peewee_plus
from .fixtures import fakedb
def test_public_api():
"""Test that the public API components are exposed via ``__all__``"""
assert peewee_plus.EnumField.__name__ in peewee_plus.__all__
def test_enum(fakedb):
"""Test basic functionality of the enum field"""

View File

@ -1,78 +0,0 @@
# pylint: disable=redefined-outer-name
# pylint: disable=missing-class-docstring
# pylint: disable=too-few-public-methods
# pylint: disable=unused-import
from pathlib import Path
import peewee
import pytest
import peewee_plus
from .fixtures import fakedb
def test_public_api():
"""Test that the public API components are exposed via ``__all__``"""
assert peewee_plus.JSONField.__name__ in peewee_plus.__all__
def test_json(fakedb):
"""Test basic usage of JSONField class"""
class TestModel(peewee.Model):
class Meta:
database = fakedb
some_data = peewee_plus.JSONField()
fakedb.create_tables([TestModel])
data = {"foo": 10, "bar": ["hello", "world"], "baz": True}
model = TestModel(some_data=data)
model.save()
model = TestModel.get()
assert model.some_data == data
def test_errors(fakedb):
"""Test that errors are raised as expected"""
class GoodModel(peewee.Model):
class Meta:
database = fakedb
# Needs to match the table name below
table_name = "one_table"
id = peewee.AutoField()
some_data = peewee_plus.JSONField()
class BadModel(peewee.Model):
class Meta:
database = fakedb
# Needs to match the table name above
table_name = "one_table"
id = peewee.AutoField()
some_data = peewee.TextField()
fakedb.create_tables([GoodModel])
with pytest.raises(ValueError):
# The usage of path here is arbitrary, it just needs to be any
# non-JSON serializable type
bad = GoodModel(some_data=Path("."))
bad.save()
good = GoodModel(some_data={"foo": 123})
good.save()
# This is overwriting the ``some_data`` on the above object with garbage
bad = BadModel.get(good.id)
bad.some_data = "This{ string' is not, valid JSON;"
bad.save()
with pytest.raises(peewee.IntegrityError):
GoodModel.get(good.id)

35
tests/test_jsonfield.py Normal file
View File

@ -0,0 +1,35 @@
# pylint: disable=redefined-outer-name
# pylint: disable=missing-class-docstring
# pylint: disable=too-few-public-methods
# pylint: disable=unused-import
from pathlib import Path
import peewee
import pytest
import peewee_plus
from .fixtures import fakedb
def test_json(fakedb):
"""Test basic usage of JSONField class"""
class TestModel(peewee.Model):
class Meta:
database = fakedb
some_data = peewee_plus.JSONField()
fakedb.create_tables([TestModel])
data = {"foo": 10, "bar": ["hello", "world"], "baz": True}
model = TestModel(some_data=data)
model.save()
model = TestModel.get()
assert model.some_data == data
with pytest.raises(peewee.IntegrityError):
bad = TestModel(some_data=Path("."))
bad.save()

View File

@ -10,12 +10,6 @@ import peewee_plus
from .fixtures import fakedb
def test_public_api():
"""Test that the public API components are exposed via ``__all__``"""
assert peewee_plus.PathField.__name__ in peewee_plus.__all__
def test_conversion(fakedb):
"""Test basic usage of PathField for roundtrip compatibility"""

View File

@ -8,12 +8,6 @@ import peewee_plus
from .fixtures import fakedb
def test_public_api():
"""Test that the public API components are exposed via ``__all__``"""
assert peewee_plus.PrecisionFloatField.__name__ in peewee_plus.__all__
# There isn't anything we can really test here since this field implements
# a MySQL-specific syntax and we test with SQLite. This test is here just
# to ensure that the behavior is consistent with the normal FloatField when

View File

@ -1,37 +0,0 @@
# pylint: disable=redefined-outer-name
# pylint: disable=missing-class-docstring
# pylint: disable=too-few-public-methods
# pylint: disable=unused-import
import datetime
from pathlib import Path
import peewee
import peewee_plus
from .fixtures import fakedb
def test_public_api():
"""Test that the public API components are exposed via ``__all__``"""
assert peewee_plus.TimedeltaField.__name__ in peewee_plus.__all__
def test_conversion(fakedb):
"""Test basic usage of PathField for roundtrip compatibility"""
class TestModel(peewee.Model):
class Meta:
database = fakedb
name = peewee.CharField()
some_timedelta = peewee_plus.TimedeltaField()
fakedb.create_tables([TestModel])
delta = datetime.timedelta(seconds=300)
model = TestModel(name="one", some_timedelta=delta)
model.save()
new = TestModel.get(TestModel.name == "one")
assert new.some_timedelta == delta

47
tox.ini
View File

@ -1,6 +1,6 @@
[tox]
envlist =
py{37,38,39,310,311}
py{37,38,39,310}
static
static-tests
security
@ -11,8 +11,11 @@ skip_missing_interpreters = true
description = Run the tests
require_locked_deps = true
require_poetry = true
poetry_dep_groups =
test
locked_deps =
pytest
pytest-cov
ruamel.yaml
toml
commands =
pytest {toxinidir}/tests/ \
--cov peewee_plus \
@ -23,8 +26,16 @@ commands =
description = Static formatting and quality enforcement
basepython = python3.10
ignore_errors = true
poetry_dep_groups =
static
locked_deps =
black
blacken-docs
mdformat
mdformat-gfm
mypy
reorder-python-imports
pre-commit
pre-commit-hooks
pylint
commands =
pre-commit run \
--all-files
@ -38,9 +49,11 @@ commands =
description = Static formatting and quality enforcement for the tests
basepython = python3.10
ignore_errors = true
poetry_dep_groups =
static
test
locked_deps =
mypy
pylint
pytest
types-toml
commands =
pylint {toxinidir}/tests/ \
--rcfile {toxinidir}/.pylintrc
@ -53,8 +66,10 @@ description = Security checks
basepython = python3.10
skip_install = true
ignore_errors = true
poetry_dep_groups =
security
locked_deps =
bandit
safety
poetry
commands =
bandit {toxinidir}/peewee_plus.py \
--recursive \
@ -67,13 +82,7 @@ commands =
--format requirements.txt \
--output {envtmpdir}/requirements.txt \
--without-hashes \
--with ci \
--with test \
--with security \
--with static \
--with dev
--dev
safety check \
--file {envtmpdir}/requirements.txt \
--output text \
# https://github.com/pytest-dev/py/issues/287
--ignore 51457
--json \
--file {envtmpdir}/requirements.txt