scrapy.utils.url.add_http_if_no_scheme

Here are the examples of the python api scrapy.utils.url.add_http_if_no_scheme taken from open source projects. By voting up you can indicate which examples are most useful and appropriate.

26 Examples 7

Example 1

Project: scrapy Source File: test_utils_url.py
    def test_add_scheme(self):
        self.assertEqual(add_http_if_no_scheme('www.example.com'),
                                               'http://www.example.com')

Example 2

Project: scrapy Source File: test_utils_url.py
    def test_without_subdomain(self):
        self.assertEqual(add_http_if_no_scheme('example.com'),
                                               'http://example.com')

Example 3

Project: scrapy Source File: test_utils_url.py
Function: test_path
    def test_path(self):
        self.assertEqual(add_http_if_no_scheme('www.example.com/some/page.html'),
                                               'http://www.example.com/some/page.html')

Example 4

Project: scrapy Source File: test_utils_url.py
Function: test_port
    def test_port(self):
        self.assertEqual(add_http_if_no_scheme('www.example.com:80'),
                                               'http://www.example.com:80')

Example 5

Project: scrapy Source File: test_utils_url.py
Function: test_fragment
    def test_fragment(self):
        self.assertEqual(add_http_if_no_scheme('www.example.com/some/page#frag'),
                                               'http://www.example.com/some/page#frag')

Example 6

Project: scrapy Source File: test_utils_url.py
Function: test_query
    def test_query(self):
        self.assertEqual(add_http_if_no_scheme('www.example.com/do?a=1&b=2&c=3'),
                                               'http://www.example.com/do?a=1&b=2&c=3')

Example 7

Project: scrapy Source File: test_utils_url.py
Function: test_username_password
    def test_username_password(self):
        self.assertEqual(add_http_if_no_scheme('username:[email protected]'),
                                               'http://username:[email protected]')

Example 8

Project: scrapy Source File: test_utils_url.py
Function: test_complete_url
    def test_complete_url(self):
        self.assertEqual(add_http_if_no_scheme('username:[email protected]:80/some/page/do?a=1&b=2&c=3#frag'),
                                               'http://username:[email protected]:80/some/page/do?a=1&b=2&c=3#frag')

Example 9

Project: scrapy Source File: test_utils_url.py
    def test_preserve_http(self):
        self.assertEqual(add_http_if_no_scheme('http://www.example.com'),
                                               'http://www.example.com')

Example 10

Project: scrapy Source File: test_utils_url.py
    def test_preserve_http_without_subdomain(self):
        self.assertEqual(add_http_if_no_scheme('http://example.com'),
                                               'http://example.com')

Example 11

Project: scrapy Source File: test_utils_url.py
    def test_preserve_http_path(self):
        self.assertEqual(add_http_if_no_scheme('http://www.example.com/some/page.html'),
                                               'http://www.example.com/some/page.html')

Example 12

Project: scrapy Source File: test_utils_url.py
    def test_preserve_http_port(self):
        self.assertEqual(add_http_if_no_scheme('http://www.example.com:80'),
                                               'http://www.example.com:80')

Example 13

Project: scrapy Source File: test_utils_url.py
    def test_preserve_http_fragment(self):
        self.assertEqual(add_http_if_no_scheme('http://www.example.com/some/page#frag'),
                                               'http://www.example.com/some/page#frag')

Example 14

Project: scrapy Source File: test_utils_url.py
    def test_preserve_http_query(self):
        self.assertEqual(add_http_if_no_scheme('http://www.example.com/do?a=1&b=2&c=3'),
                                               'http://www.example.com/do?a=1&b=2&c=3')

Example 15

Project: scrapy Source File: test_utils_url.py
    def test_preserve_http_username_password(self):
        self.assertEqual(add_http_if_no_scheme('http://username:[email protected]'),
                                               'http://username:[email protected]')

Example 16

Project: scrapy Source File: test_utils_url.py
    def test_preserve_http_complete_url(self):
        self.assertEqual(add_http_if_no_scheme('http://username:[email protected]:80/some/page/do?a=1&b=2&c=3#frag'),
                                               'http://username:[email protected]:80/some/page/do?a=1&b=2&c=3#frag')

Example 17

Project: scrapy Source File: test_utils_url.py
Function: test_protocol_relative
    def test_protocol_relative(self):
        self.assertEqual(add_http_if_no_scheme('//www.example.com'),
                                               'http://www.example.com')

Example 18

Project: scrapy Source File: test_utils_url.py
    def test_protocol_relative_without_subdomain(self):
        self.assertEqual(add_http_if_no_scheme('//example.com'),
                                               'http://example.com')

Example 19

Project: scrapy Source File: test_utils_url.py
    def test_protocol_relative_path(self):
        self.assertEqual(add_http_if_no_scheme('//www.example.com/some/page.html'),
                                               'http://www.example.com/some/page.html')

Example 20

Project: scrapy Source File: test_utils_url.py
    def test_protocol_relative_port(self):
        self.assertEqual(add_http_if_no_scheme('//www.example.com:80'),
                                               'http://www.example.com:80')

Example 21

Project: scrapy Source File: test_utils_url.py
    def test_protocol_relative_fragment(self):
        self.assertEqual(add_http_if_no_scheme('//www.example.com/some/page#frag'),
                                               'http://www.example.com/some/page#frag')

Example 22

Project: scrapy Source File: test_utils_url.py
    def test_protocol_relative_query(self):
        self.assertEqual(add_http_if_no_scheme('//www.example.com/do?a=1&b=2&c=3'),
                                               'http://www.example.com/do?a=1&b=2&c=3')

Example 23

Project: scrapy Source File: test_utils_url.py
    def test_protocol_relative_username_password(self):
        self.assertEqual(add_http_if_no_scheme('//username:[email protected]'),
                                               'http://username:[email protected]')

Example 24

Project: scrapy Source File: test_utils_url.py
    def test_protocol_relative_complete_url(self):
        self.assertEqual(add_http_if_no_scheme('//username:[email protected]:80/some/page/do?a=1&b=2&c=3#frag'),
                                               'http://username:[email protected]:80/some/page/do?a=1&b=2&c=3#frag')

Example 25

Project: scrapy Source File: test_utils_url.py
    def test_preserve_https(self):
        self.assertEqual(add_http_if_no_scheme('https://www.example.com'),
                                               'https://www.example.com')

Example 26

Project: scrapy Source File: test_utils_url.py
    def test_preserve_ftp(self):
        self.assertEqual(add_http_if_no_scheme('ftp://www.example.com'),
                                               'ftp://www.example.com')