grab.transport

Module Contents

Classes

Urllib3Transport

Grab network transport based on urllib3 library.

Attributes

LOG

grab.transport.LOG

class grab.transport.Urllib3Transport

Bases: grab.base.BaseTransport[grab.request.HttpRequest, grab.document.Document]

Grab network transport based on urllib3 library.

__getstate__() → dict[str, Any]

__setstate__(state: collections.abc.Mapping[str, Any]) → None

build_pool() → urllib3.PoolManager

reset() → None

wrap_transport_error() → collections.abc.Generator[None, None, None]

select_pool_for_request(req: grab.request.HttpRequest) → urllib3.PoolManager | urllib3.ProxyManager | urllib3.contrib.socks.SOCKSProxyManager

log_request(req: grab.request.HttpRequest) → None

Log request details via logging system.

request(req: grab.request.HttpRequest) → None

read_with_timeout(req: grab.request.HttpRequest) → bytes

get_response_header_items() → list[tuple[str, Any]]

Return current response headers as items.

This funciton is required to isolated smalles part of untyped code and hide it from mypy

prepare_response(req: grab.request.HttpRequest, *, document_class: type[grab.document.Document] = Document) → grab.document.Document

Prepare response, duh.

This methed is called after network request is completed hence the “self._request” is not None.

Good to know: on python3 urllib3 headers are converted to str type using latin encoding.