pax_global_header00006660000000000000000000000064140621711350014512gustar00rootroot0000000000000052 comment=2abcc1931a2f01b49841412d8d204379d292cfb2 ods-pytest-logdog-b8acbf0/000077500000000000000000000000001406217113500156455ustar00rootroot00000000000000ods-pytest-logdog-b8acbf0/.gitignore000066400000000000000000000001031406217113500176270ustar00rootroot00000000000000__pycache__/ /.tox/ /.coverage /.eggs/ /*.egg-info/ /build/ /dist/ ods-pytest-logdog-b8acbf0/CHANGELOG.md000066400000000000000000000000531406217113500174540ustar00rootroot00000000000000# Change Log ## 0.1.0 (2021-06-15) * MVP ods-pytest-logdog-b8acbf0/DESIGN.md000066400000000000000000000043251406217113500171440ustar00rootroot00000000000000# Rationale and design ## What's wrong with caplog? There is no ready methods to filter interesting records. It creates a mess from captured record at different stages. `get_records()` allows to access records for the given stage, but `clear()` affects all. `caplog.at_level()` / `caplog.set_level()` changes the state globally, so it's not possible to structure your tools without the risk of interference. Even worse, using `logger` argument might prevent you from seeing some records in root logger. There is not way to disable output of handled messages. You watch a ton of messages without a chance to distinguish whether some `ERROR` is a deliberate while testing corner cases, or unexpected and requires your attention. ## Requirements 1. (high/simple) Simple interface to match records similar to filters: by logger name, level, message pattern. 2. (medium/simple) Some way to avoid interference of test parts. For example, scopes to narrow what we capture. Allow nesting if possible. 3. (medium/hard) Clean output: allow marking records as handled in some way ("pop them out") and don't show them in the output. 4. (low/hard) Option to switch whether to hide (default) or show handled messages in the output. 5. (low/hard) Visualy mark handled messages in the output, when you opted to show them. ## Ways to implement ### Wrapper around `caplog` pros: * Easiest way to reach requirement #1. cons: * No forseeable ways to get scoping and meet other requirements. ### Install single handler to root logger pros: * With single handler it should be easier to collect/mark handled messages (requirement #3). cons: * It's tricky to separate messages for each scope. * Some loggers may have level set and thus fall out of consideration. ### Install separate handler for each scope pros: * Easy separation of messages for each scope. * It's possible to attach capturer to specific logger and reset level for it. cons: * We need some additional global entity to collect/mark handled messages. ## Choosing the name The current working name (`xcaplog`) is great for wrapper implementation, as it extends the original `caplog`. But all other ways to implement it lead to quite different API and thus connection to `caplog` is arguable. ods-pytest-logdog-b8acbf0/LICENSE000066400000000000000000000020571406217113500166560ustar00rootroot00000000000000MIT License Copyright (c) 2021 Denis Otkidach Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ods-pytest-logdog-b8acbf0/README.md000066400000000000000000000007421406217113500171270ustar00rootroot00000000000000# pytest-logdog: Pytest plugin to test logging ## Usage ```python pytest_plugins = ["pytest_logdog"] def test_it_works(logdog): with logdog() as pile: logging.info("Hello world!") [rec] = pile.drain(message="Hello.*") assert rec.levelno == logging.INFO assert pile.is_empty() ``` ## Links * [Rationale and design](https://github.com/ods/pytest-logdog/blob/master/DESIGN.md) * [Change log](https://github.com/ods/pytest-logdog/blob/master/CHANGELOG.md) ods-pytest-logdog-b8acbf0/TODO.md000066400000000000000000000022631406217113500167370ustar00rootroot00000000000000# To-do list (temporary) All items here are subject for discussion. ## Features ### Filters * [ ] A way to filter by exact value of `level`? `level` + `level_exact`? or `level_ge` + `level_eq`?. ### Other * [ ] `assert_one_pop()` to pop matching and assert only one. Is it possible to provide custome error message with similar records? Or full list if it has limitted size. * [ ] Add methods like `get_text()` to `LogPile` (note, it's also returned from `filter` and `drain`)? * [ ] Some way to automate `assert pile.is_empty()`? In `__exit__`? * [ ] Capture all by default for root (i.e. reset to `NOTSET`)? If so, the fixture itself should have pile interface. * [ ] Return `LogDog` instance from fixture and provide `__call__` method? This would simplfy annotation (`def test_smth(logdog: LogDog)` instead of current `def test_smth(logdog: Type[LogDog])`), but it also allow undesirable `with logdog as pile`. Rename `LogDog` to `LogDocContext` and define `LogDog` with `__call__`? Export `LogDog` in top-level package to allow `from pytest_logdog import LogDog`? Or may be enter context and return `Pile` from fixture and provide `__call__` method in it to allow using without `with`? ods-pytest-logdog-b8acbf0/pytest_logdog/000077500000000000000000000000001406217113500205305ustar00rootroot00000000000000ods-pytest-logdog-b8acbf0/pytest_logdog/__init__.py000066400000000000000000000002431406217113500226400ustar00rootroot00000000000000from pkg_resources import get_distribution, DistributionNotFound try: __version__ = get_distribution(__name__).version except DistributionNotFound: pass ods-pytest-logdog-b8acbf0/pytest_logdog/plugin.py000066400000000000000000000135741406217113500224120ustar00rootroot00000000000000import logging import re from typing import Callable, Iterable, List, Optional, Tuple, Type, Union import pytest RecordFilter = Callable[[logging.LogRecord], bool] ExceptionType = Union[Type[BaseException], Tuple[Type[BaseException], ...]] def _get_log_level_no(level: Union[int, str]) -> int: return logging._checkLevel(level) # type: ignore class LogPile: __slots__ = ("_records",) _records: List[logging.LogRecord] def __init__(self, records: Iterable[logging.LogRecord] = ()): self._records = list(records) def __len__(self): return len(self._records) def __iter__(self): return iter(self._records) # `assert pile.is_empty()` is more readable than `assert not pile` def is_empty(self) -> bool: """Return True if the pile is empty, False otherwise.""" return not self._records def _add(self, record): self._records.append(record) def messages(self) -> List[str]: return [record.getMessage() for record in self._records] def _partition( self, func: Optional[RecordFilter] = None, *, name: Optional[str] = None, level: Union[None, int, str] = None, message: Optional[str] = None, exc_info: Union[None, bool, ExceptionType] = None, stack_info: Optional[bool] = None, ) -> Tuple[List[logging.LogRecord], List[logging.LogRecord]]: filters = [] if func is not None: filters.append(func) if name is not None: def _filter(record): # The same behavior as for `logging.Filter(name=...)` return ( record.name == name or record.name.startswith(f"{name}.") ) filters.append(_filter) if level is not None: levelno = _get_log_level_no(level) if levelno: def _filter(record): return record.levelno >= levelno filters.append(_filter) if message is not None: def _filter(record): return re.search(message, record.getMessage()) filters.append(_filter) if exc_info is not None: if isinstance(exc_info, (type, tuple)): exc_type = exc_info def _filter(record): return ( record.exc_info is not None and isinstance(record.exc_info[1], exc_type) ) else: has_exc_info = bool(exc_info) def _filter(record): return (record.exc_info is not None) == has_exc_info filters.append(_filter) if stack_info is not None: has_stack_info = bool(stack_info) def _filter(record): return (record.stack_info is not None) == has_stack_info filters.append(_filter) matching = [] rest = [] for record in self._records: if all(matches(record) for matches in filters): matching.append(record) else: rest.append(record) return matching, rest def filter( self, func: Optional[RecordFilter] = None, *, name: Optional[str] = None, level: Union[None, int, str] = None, message: Optional[str] = None, exc_info: Union[None, bool, ExceptionType] = None, stack_info: Optional[bool] = None, ) -> "LogPile": """Return list of matching log records.""" matching, _ = self._partition( func, name=name, level=level, message=message, exc_info=exc_info, stack_info=stack_info ) return LogPile(matching) def drain( self, func: Optional[RecordFilter] = None, *, name: Optional[str] = None, level: Union[None, int, str] = None, message: Optional[str] = None, exc_info: Union[None, bool, ExceptionType] = None, stack_info: Optional[bool] = None, ) -> "LogPile": """Return list of matching log records and remove them from the pile. """ matching, rest = self._partition( func, name=name, level=level, message=message, exc_info=exc_info, stack_info=stack_info ) # Atomically update without locks count = len(matching) + len(rest) self._records[:count] = rest # Commented buggy version to ensure test catches the race: # self._records = rest return LogPile(matching) class LogHandler(logging.Handler): __slots__ = ("_pile",) def __init__(self, pile): super().__init__() self._pile = pile def handle(self, record): self._pile._records.append(record) class LogDog: __slots__ = ("_logger", "_handler", "_orig_level", "_level") def __init__( self, name: Optional[str] = None, level: Union[None, int, str] = None, ): self._logger = logging.getLogger(name) self._level = level def __enter__(self): pile = LogPile() self._handler = LogHandler(pile) if self._level is not None: self._handler.setLevel(self._level) self._logger.addHandler(self._handler) if self._level is not None: self._orig_level = self._logger.level # Argument `level` can be `None`, `int` or `str`, while # `handler.level` is always `int` (converted by `setLevel()` # method) self._logger.setLevel(min(self._orig_level, self._handler.level)) return pile def __exit__(self, type, value, traceback): if self._level is not None: self._logger.setLevel(self._orig_level) self._logger.removeHandler(self._handler) @pytest.fixture def logdog(): """Scoped log capturing and testing tool.""" return LogDog ods-pytest-logdog-b8acbf0/setup.cfg000066400000000000000000000025661406217113500174770ustar00rootroot00000000000000[metadata] name = pytest-logdog description = Pytest plugin to test logging long_description = file: README.md long_description_content_type = text/markdown author = Denis Otkidach author_email = denis.otkidach@gmail.com url = https://github.com/ods/pytest-logdog license = MIT license_file = LICENSE classifiers = Development Status :: 1 - Planning Intended Audience :: Developers License :: OSI Approved :: MIT License Programming Language :: Python :: 3.7 Programming Language :: Python :: 3.8 Programming Language :: Python :: 3.9 Topic :: Software Development :: Testing Framework :: Pytest [options] packages = pytest_logdog python_requires = >=3.7 install_requires = pytest>=6.2.0 setup_requires = setuptools-scm>=6.0.1 [options.entry_points] pytest11 = logdog = pytest_logdog.plugin [tool:pytest] testpaths = tests.py addopts = --strict-markers -r aP --tb=native filterwarnings = error [coverage:run] branch = True source = . omit = setup.py tests/* .tox/* [coverage:report] show_missing = True [mypy] ignore_missing_imports = True check_untyped_defs = True warn_redundant_casts = True warn_unused_ignores = True [tox:tox] envlist = py{37,38,39},mypy [testenv] deps = pytest-cov commands = pytest {posargs:--cov --cov-report=} [testenv:mypy] basepython = python3.9 deps = mypy>=0.812 commands = mypy -p pytest_logdog -p tests ods-pytest-logdog-b8acbf0/setup.py000066400000000000000000000000731406217113500173570ustar00rootroot00000000000000from setuptools import setup setup(use_scm_version=True) ods-pytest-logdog-b8acbf0/tests.py000066400000000000000000000135371406217113500173720ustar00rootroot00000000000000from concurrent.futures import ThreadPoolExecutor import logging from random import random from time import sleep import pytest pytest_plugins = ["pytest_logdog"] def test_it_works(logdog): with logdog(level=logging.INFO) as pile: logging.info("Test") assert len(pile) == 1 [rec] = pile assert rec.getMessage() == "Test" def test_log_drain_race(logdog): COUNT = 100 def log(): for _ in range(COUNT): sleep(random() * 0.000_001) logging.info("Test") drained = [] with ThreadPoolExecutor() as executor, logdog(level=logging.INFO) as pile: future = executor.submit(log) while not future.done(): drained.extend(pile.drain()) drained.extend(pile.drain()) assert len(drained) == COUNT def test_nested(logdog): with logdog(level=logging.WARNING) as outer: logging.warning("Outer warning 1") with logdog(level=logging.INFO) as inner: logging.info("Inner info") logging.warning("Inner warning") logging.warning("Outer warning 2") assert outer.messages() == [ "Outer warning 1", "Inner warning", "Outer warning 2", ] assert inner.messages() == [ "Inner info", "Inner warning", ] def test_preset_level(logdog): logger = logging.getLogger("mod") logger.setLevel(logging.WARNING) # Precondition: without it the test in the next block doesn't make sense with logdog(level=logging.INFO) as pile: logger.info("Silenced") assert pile.is_empty() with logdog(name="mod", level=logging.INFO) as pile: logger.info("Aloud") assert pile.messages() == ["Aloud"] @pytest.mark.parametrize( "name, matches", [("", False), ("mod", True), ("module", False), ("mod.sub", True)], ) def test_capture_name(logdog, name, matches): with logdog(name="mod") as pile: logging.getLogger(name).error("Message") assert pile.is_empty() == (not matches) @pytest.mark.parametrize( "name, matches", [("", False), ("mod", True), ("module", False), ("mod.sub", True)], ) def test_filter_drain_name(logdog, name, matches): with logdog() as pile: logging.getLogger(name).error("Message") assert pile.filter(name="mod").is_empty() == (not matches) assert not pile.is_empty() assert pile.drain(name="mod").is_empty() == (not matches) assert pile.is_empty() == matches @pytest.mark.parametrize( "log_level, filter_level, matches", [ (logging.DEBUG, logging.INFO, False), (logging.DEBUG, logging.DEBUG, True), (logging.DEBUG, logging.NOTSET, True), (logging.DEBUG, "DEBUG", True), (logging.DEBUG, 5, True), (logging.DEBUG, 15, False), ], ) def test_filter_drain_level(logdog, log_level, filter_level, matches): with logdog(level=logging.NOTSET) as pile: logging.log(log_level, "Message") assert pile.filter(level=filter_level).is_empty() == (not matches) assert not pile.is_empty() assert pile.drain(level=filter_level).is_empty() == (not matches) assert pile.is_empty() == matches @pytest.mark.parametrize( "pattern, matches", [ ("^one", True), ("two", True), ("^two", False), ("one.*three", True), ], ) def test_filter_drain_message(logdog, pattern, matches): with logdog() as pile: logging.error("one two three") assert pile.filter(message=pattern).is_empty() == (not matches) assert not pile.is_empty() assert pile.drain(message=pattern).is_empty() == (not matches) assert pile.is_empty() == matches @pytest.mark.parametrize( "exc_info, matches", [ (None, True), (False, False), (True, True), (0, False), (1, True), (object(), True), (ZeroDivisionError, True), (Exception, True), (RuntimeError, False), ((ValueError, ArithmeticError), True), ((ValueError, TypeError), False), ], ) def test_filter_drain_exc_info_exception(logdog, exc_info, matches): with logdog() as pile: try: 1 / 0 except: logging.exception("Error") assert pile.filter(exc_info=exc_info).is_empty() == (not matches) assert not pile.is_empty() assert pile.drain(exc_info=exc_info).is_empty() == (not matches) assert pile.is_empty() == matches @pytest.mark.parametrize( "exc_info, matches", [ (None, True), (False, True), (True, False), (0, True), (1, False), (object(), False), (Exception, False), ((ValueError, TypeError), False), ], ) def test_filter_drain_exc_info_no_exception(logdog, exc_info, matches): with logdog() as pile: logging.error("Error") assert pile.filter(exc_info=exc_info).is_empty() == (not matches) assert not pile.is_empty() assert pile.drain(exc_info=exc_info).is_empty() == (not matches) assert pile.is_empty() == matches @pytest.mark.parametrize( "log_stack_info, filter_stack_info, matches", [ (None, None, True), (False, None, True), (True, None, True), (None, False, True), (None, True, False), (False, False, True), (False, 0, True), (False, True, False), (False, 1, False), (False, object(), False), (True, False, False), (True, 0, False), (True, True, True), (True, 1, True), (True, object(), True), ], ) def test_filter_drain_stack_info( logdog, log_stack_info, filter_stack_info, matches ): with logdog() as pile: logging.error("Error", stack_info=log_stack_info) assert pile.filter(stack_info=filter_stack_info).is_empty() == (not matches) assert not pile.is_empty() assert pile.drain(stack_info=filter_stack_info).is_empty() == (not matches) assert pile.is_empty() == matches