scrapi.settings.NORMALIZED_PROCESSING

Here are the examples of the python api scrapi.settings.NORMALIZED_PROCESSING taken from open source projects. By voting up you can indicate which examples are most useful and appropriate.

2 Examples 7

Example 1

Project: scrapi Source File: __init__.py
Function: process_normalized
def process_normalized(raw_doc, normalized, kwargs):
    ''' kwargs is a dictiorary of kwargs.
        keyed by the processor name
        Exists so that when we run check archive we
        specifiy that it's ok to overrite certain files
    '''
    assert (raw_doc and normalized), 'Raw and normalized docuements must be provided to process_normalized'
    for p in settings.NORMALIZED_PROCESSING:
        extras = kwargs.get(p, {})
        get_processor(p).process_normalized(raw_doc, normalized, **extras)

Example 2

Project: scrapi Source File: test_processing.py
def test_normalized_calls_all(get_processor):
    processing.process_normalized(mock.MagicMock(), mock.MagicMock(), {})

    for processor in settings.NORMALIZED_PROCESSING:
        get_processor.assert_any_call(processor)