Here are the examples of the python api scrapi.settings.NORMALIZED_PROCESSING taken from open source projects. By voting up you can indicate which examples are most useful and appropriate.
2 Examples
3
Example 1
def process_normalized(raw_doc, normalized, kwargs):
''' kwargs is a dictiorary of kwargs.
keyed by the processor name
Exists so that when we run check archive we
specifiy that it's ok to overrite certain files
'''
assert (raw_doc and normalized), 'Raw and normalized docuements must be provided to process_normalized'
for p in settings.NORMALIZED_PROCESSING:
extras = kwargs.get(p, {})
get_processor(p).process_normalized(raw_doc, normalized, **extras)
0
Example 2
Project: scrapi Source File: test_processing.py
def test_normalized_calls_all(get_processor):
processing.process_normalized(mock.MagicMock(), mock.MagicMock(), {})
for processor in settings.NORMALIZED_PROCESSING:
get_processor.assert_any_call(processor)