release 2018.05.18

[ChangeLog] Actualize
[ci skip]
2025-10-17 05:38:35 +09:00 · 2018-05-18 00:32:51 +07:00 · 2018-05-18 00:30:41 +07:00 · 2018-05-17 23:21:40 +07:00 · 2018-05-17 13:46:05 +01:00 · 2018-05-16 19:11:48 +01:00
85 changed files with 2251 additions and 911 deletions
--- a/.github/ISSUE_TEMPLATE.md
+++ b/.github/ISSUE_TEMPLATE.md
@@ -6,8 +6,8 @@

 ---

-### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2018.04.03*. If it's not, read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
- [ ] I've **verified** and **I assure** that I'm running youtube-dl **2018.04.03**
+### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2018.05.18*. If it's not, read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
+- [ ] I've **verified** and **I assure** that I'm running youtube-dl **2018.05.18**

 ### Before submitting an *issue* make sure you have:
 - [ ] At least skimmed through the [README](https://github.com/rg3/youtube-dl/blob/master/README.md), **most notably** the [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections
@@ -36,7 +36,7 @@ Add the `-v` flag to **your command line** you run youtube-dl with (`youtube-dl
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
-[debug] youtube-dl version 2018.04.03
+[debug] youtube-dl version 2018.05.18
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
--- a/3
+++ b/3
@@ -236,3 +236,6 @@ Lei Wang
 Petr Novák
 Leonardo Taccari
 Martin Weinelt
+Surya Oktafendri
+TingPing
+Alexandre Macabies
--- a/140
+++ b/140
@@ -1,3 +1,143 @@
+version 2018.05.18
+
+Extractors
+* [vimeo:likes] Relax URL regular expression and fix single page likes
+  extraction (#16475)
+* [pluralsight] Fix clip id extraction (#16460)
+ [mychannels] Add support for mychannels.com (#15334)
+- [moniker] Remove extractor (#15336)
+* [pbs] Fix embed data extraction (#16474)
+ [mtv] Add support for paramountnetwork.com and bellator.com (#15418)
+* [youtube] Fix hd720 format position
+* [dailymotion] Remove fragment part from m3u8 URLs (#8915)
+* [3sat] Improve extraction (#15350)
+    * Extract all formats
+    * Extract more format metadata
+    * Improve format sorting
+    * Use hls native downloader
+    * Detect and bypass geo-restriction
+ [dtube] Add support for d.tube (#15201)
+* [options] Fix typo (#16450)
+* [youtube] Improve format filesize extraction (#16453)
+* [youtube] Make uploader extraction non fatal (#16444)
+* [youtube] Fix extraction for embed restricted live streams (#16433)
+* [nbc] Improve info extraction (#16440)
+* [twitch:clips] Fix extraction (#16429)
+* [redditr] Relax URL regular expression (#16426, #16427)
+* [mixcloud] Bypass throttling for HTTP formats (#12579, #16424)
+ [nick] Add support for nickjr.de (#13230)
+* [teamcoco] Fix extraction (#16374)
+
+
+version 2018.05.09
+
+Core
+* [YoutubeDL] Ensure ext exists for automatic captions
+* Introduce --geo-bypass-ip-block
+
+Extractors
+ [udemy] Extract asset captions
+ [udemy] Extract stream URLs (#16372)
+ [businessinsider] Add support for businessinsider.com (#16387, #16388, #16389)
+ [cloudflarestream] Add support for cloudflarestream.com (#16375)
+* [watchbox] Fix extraction (#16356)
+* [discovery] Extract Affiliate/Anonymous Auth Token from cookies (#14954)
+ [itv:btcc] Add support for itv.com/btcc (#16139)
+* [tunein] Use live title for live streams (#16347)
+* [itv] Improve extraction (#16253)
+
+
+version 2018.05.01
+
+Core
+* [downloader/fragment] Restart download if .ytdl file is corrupt (#16312)
+ [extractor/common] Extract interaction statistic
+ [utils] Add merge_dicts
+ [extractor/common] Add _download_json_handle
+
+Extractors
+* [kaltura] Improve iframe embeds detection (#16337)
+ [udemy] Extract outputs renditions (#16289, #16291, #16320, #16321, #16334,
+  #16335)
+ [zattoo] Add support for zattoo.com and mobiltv.quickline.com (#14668, #14676)
+* [yandexmusic] Convert release_year to int
+* [udemy] Override _download_webpage_handle instead of _download_webpage
+* [xiami] Override _download_webpage_handle instead of _download_webpage
+* [yandexmusic] Override _download_webpage_handle instead of _download_webpage
+* [youtube] Correctly disable polymer on all requests (#16323, #16326)
+* [generic] Prefer enclosures over links in RSS feeds (#16189)
+ [redditr] Add support for old.reddit.com URLs (#16274)
+* [nrktv] Update API host (#16324)
+ [imdb] Extract all formats (#16249)
+ [vimeo] Extract JSON-LD (#16295)
+* [funk:channel] Improve extraction (#16285)
+
+
+version 2018.04.25
+
+Core
+* [utils] Fix match_str for boolean meta fields
+ [Makefile] Add support for pandoc 2 and disable smart extension (#16251)
+* [YoutubeDL] Fix typo in media extension compatibility checker (#16215)
+
+Extractors
+ [openload] Recognize IPv6 stream URLs (#16136, #16137, #16205, #16246,
+  #16250)
+ [twitch] Extract is_live according to status (#16259)
+* [pornflip] Relax URL regular expression (#16258)
+- [etonline] Remove extractor (#16256)
+* [breakcom] Fix extraction (#16254)
+ [youtube] Add ability to authenticate with cookies
+* [youtube:feed] Implement lazy playlist extraction (#10184)
+ [svt] Add support for TV channel live streams (#15279, #15809)
+* [ccma] Fix video extraction (#15931)
+* [rentv] Fix extraction (#15227)
+ [nick] Add support for nickjr.nl (#16230)
+* [extremetube] Fix metadata extraction
+ [keezmovies] Add support for generic embeds (#16134, #16154)
+* [nexx] Extract new azure URLs (#16223)
+* [cbssports] Fix extraction (#16217)
+* [kaltura] Improve embeds detection (#16201)
+* [instagram:user] Fix extraction (#16119)
+* [cbs] Skip DRM asset types (#16104)
+
+
+version 2018.04.16
+
+Extractors
+* [smotri:broadcast] Fix extraction (#16180)
+ [picarto] Add support for picarto.tv (#6205, #12514, #15276, #15551)
+* [vine:user] Fix extraction (#15514, #16190)
+* [pornhub] Relax URL regular expression (#16165)
+* [cbc:watch] Re-acquire device token when expired (#16160)
+ [fxnetworks] Add support for https theplatform URLs (#16125, #16157)
+ [instagram:user] Add request signing (#16119)
+ [twitch] Add support for mobile URLs (#16146)
+
+
+version 2018.04.09
+
+Core
+* [YoutubeDL] Do not save/restore console title while simulate (#16103)
+* [extractor/common] Relax JSON-LD context check (#16006)
+
+Extractors
+ [generic] Add support for tube8 embeds
+ [generic] Add support for share-videos.se embeds (#16089, #16115)
+* [odnoklassniki] Extend URL regular expression (#16081)
+* [steam] Bypass mature content check (#16113)
+ [acast] Extract more metadata
+* [acast] Fix extraction (#16118)
+* [instagram:user] Fix extraction (#16119)
+* [drtuber] Fix title extraction (#16107, #16108)
+* [liveleak] Extend URL regular expression (#16117)
+ [openload] Add support for oload.xyz
+* [openload] Relax stream URL regular expression
+* [openload] Fix extraction (#16099)
+ [svtplay:series] Add support for season URLs
+ [svtplay:series] Add support for series (#11130, #16059)
+
+
 version 2018.04.03

 Extractors
--- a/7
+++ b/7
@@ -14,6 +14,9 @@ PYTHON ?= /usr/bin/env python
 # set SYSCONFDIR to /etc if PREFIX=/usr or PREFIX=/usr/local
 SYSCONFDIR = $(shell if [ $(PREFIX) = /usr -o $(PREFIX) = /usr/local ]; then echo /etc; else echo $(PREFIX)/etc; fi)

+# set markdown input format to "markdown-smart" for pandoc version 2 and to "markdown" for pandoc prior to version 2
+MARKDOWN = $(shell if [ `pandoc -v | head -n1 | cut -d" " -f2 | head -c1` = "2" ]; then echo markdown-smart; else echo markdown; fi)
+
 install: youtube-dl youtube-dl.1 youtube-dl.bash-completion youtube-dl.zsh youtube-dl.fish
 	install -d $(DESTDIR)$(BINDIR)
 	install -m 755 youtube-dl $(DESTDIR)$(BINDIR)
@@ -82,11 +85,11 @@ supportedsites:
 	$(PYTHON) devscripts/make_supportedsites.py docs/supportedsites.md

 README.txt: README.md
-	pandoc -f markdown -t plain README.md -o README.txt
+	pandoc -f $(MARKDOWN) -t plain README.md -o README.txt

 youtube-dl.1: README.md
 	$(PYTHON) devscripts/prepare_manpage.py youtube-dl.1.temp.md
-	pandoc -s -f markdown -t man youtube-dl.1.temp.md -o youtube-dl.1
+	pandoc -s -f $(MARKDOWN) -t man youtube-dl.1.temp.md -o youtube-dl.1
 	rm -f youtube-dl.1.temp.md

 youtube-dl.bash-completion: youtube_dl/*.py youtube_dl/*/*.py devscripts/bash-completion.in
--- a/README.md
+++ b/README.md
@@ -106,7 +106,7 @@ Alternatively, refer to the [developer instructions](#developer-instructions) fo
    --geo-verification-proxy URL     Use this proxy to verify the IP address for
                                     some geo-restricted sites. The default
                                     proxy specified by --proxy (or none, if the
-                                     options is not present) is used for the
+                                     option is not present) is used for the
                                     actual downloading.
    --geo-bypass                     Bypass geographic restriction via faking
                                     X-Forwarded-For HTTP header (experimental)
@@ -116,6 +116,9 @@ Alternatively, refer to the [developer instructions](#developer-instructions) fo
    --geo-bypass-country CODE        Force bypass geographic restriction with
                                     explicitly provided two-letter ISO 3166-2
                                     country code (experimental)
+    --geo-bypass-ip-block IP_BLOCK   Force bypass geographic restriction with
+                                     explicitly provided IP block in CIDR
+                                     notation (experimental)

 ## Video Selection:
    --playlist-start NUMBER          Playlist video to start at (default is 1)
--- a/devscripts/gh-pages/generate-download.py
+++ b/devscripts/gh-pages/generate-download.py
@@ -1,27 +1,22 @@
 #!/usr/bin/env python3
 from __future__ import unicode_literals

-import hashlib
-import urllib.request
 import json

 versions_info = json.load(open('update/versions.json'))
 version = versions_info['latest']
-URL = versions_info['versions'][version]['bin'][0]
-
-data = urllib.request.urlopen(URL).read()
+version_dict = versions_info['versions'][version]

 # Read template page
 with open('download.html.in', 'r', encoding='utf-8') as tmplf:
    template = tmplf.read()

-sha256sum = hashlib.sha256(data).hexdigest()
 template = template.replace('@PROGRAM_VERSION@', version)
-template = template.replace('@PROGRAM_URL@', URL)
-template = template.replace('@PROGRAM_SHA256SUM@', sha256sum)
-template = template.replace('@EXE_URL@', versions_info['versions'][version]['exe'][0])
-template = template.replace('@EXE_SHA256SUM@', versions_info['versions'][version]['exe'][1])
-template = template.replace('@TAR_URL@', versions_info['versions'][version]['tar'][0])
-template = template.replace('@TAR_SHA256SUM@', versions_info['versions'][version]['tar'][1])
+template = template.replace('@PROGRAM_URL@', version_dict['bin'][0])
+template = template.replace('@PROGRAM_SHA256SUM@', version_dict['bin'][1])
+template = template.replace('@EXE_URL@', version_dict['exe'][0])
+template = template.replace('@EXE_SHA256SUM@', version_dict['exe'][1])
+template = template.replace('@TAR_URL@', version_dict['tar'][0])
+template = template.replace('@TAR_SHA256SUM@', version_dict['tar'][1])
 with open('download.html', 'w', encoding='utf-8') as dlf:
    dlf.write(template)
--- a/docs/supportedsites.md
+++ b/docs/supportedsites.md
@@ -100,6 +100,7 @@
 - **Beatport**
 - **Beeg**
 - **BehindKink**
+ - **Bellator**
 - **BellMedia**
 - **Bet**
 - **Bigflix**
@@ -122,6 +123,7 @@
 - **BRMediathek**: Bayerischer Rundfunk Mediathek
 - **bt:article**: Bergens Tidende Articles
 - **bt:vestlendingen**: Bergens Tidende - Vestlendingen
+ - **BusinessInsider**
 - **BuzzFeed**
 - **BYUtv**
 - **Camdemy**
@@ -163,6 +165,7 @@
 - **ClipRs**
 - **Clipsyndicate**
 - **CloserToTruth**
+ - **CloudflareStream**
 - **cloudtime**: CloudTime
 - **Cloudy**
 - **Clubic**
@@ -232,6 +235,7 @@
 - **DrTuber**
 - **drtv**
 - **drtv:live**
+ - **DTube**
 - **Dumpert**
 - **dvtv**: http://video.aktualne.cz/
 - **dw**
@@ -257,7 +261,6 @@
 - **ESPN**
 - **ESPNArticle**
 - **EsriVideo**
- - **ETOnline**
 - **Europa**
 - **EveryonesMixtape**
 - **ExpoTV**
@@ -374,6 +377,7 @@
 - **Ir90Tv**
 - **ITTF**
 - **ITV**
+ - **ITVBTCC**
 - **ivi**: ivi.ru
 - **ivi:compilation**: ivi.ru compilations
 - **ivideon**: Ivideon TV
@@ -446,7 +450,6 @@
 - **mailru**: Видео@Mail.Ru
 - **mailru:music**: Музыка@Mail.Ru
 - **mailru:music:search**: Музыка@Mail.Ru
- - **MakersChannel**
 - **MakerTV**
 - **mangomolo:live**
 - **mangomolo:video**
@@ -484,7 +487,6 @@
 - **MoeVideo**: LetitBit video services: moevideo.net, playreplay.net and videochart.net
 - **Mofosex**
 - **Mojvideo**
- - **Moniker**: allmyvideos.net and vidspot.net
 - **Morningstar**: morningstar.com
 - **Motherless**
 - **MotherlessGroup**
@@ -506,6 +508,7 @@
 - **mva:course**: Microsoft Virtual Academy courses
 - **Mwave**
 - **MwaveMeetGreet**
+ - **MyChannels**
 - **MySpace**
 - **MySpace:album**
 - **MySpass**
@@ -616,6 +619,7 @@
 - **PacktPubCourse**
 - **PandaTV**: 熊猫TV
 - **pandora.tv**: 판도라TV
+ - **ParamountNetwork**
 - **parliamentlive.tv**: UK parliament videos
 - **Patreon**
 - **pbs**: Public Broadcasting Service (PBS) and member stations: PBS: Public Broadcasting Service, APT - Alabama Public Television (WBIQ), GPB/Georgia Public Broadcasting (WGTV), Mississippi Public Broadcasting (WMPN), Nashville Public Television (WNPT), WFSU-TV (WFSU), WSRE (WSRE), WTCI (WTCI), WPBA/Channel 30 (WPBA), Alaska Public Media (KAKM), Arizona PBS (KAET), KNME-TV/Channel 5 (KNME), Vegas PBS (KLVX), AETN/ARKANSAS ETV NETWORK (KETS), KET (WKLE), WKNO/Channel 10 (WKNO), LPB/LOUISIANA PUBLIC BROADCASTING (WLPB), OETA (KETA), Ozarks Public Television (KOZK), WSIU Public Broadcasting (WSIU), KEET TV (KEET), KIXE/Channel 9 (KIXE), KPBS San Diego (KPBS), KQED (KQED), KVIE Public Television (KVIE), PBS SoCal/KOCE (KOCE), ValleyPBS (KVPT), CONNECTICUT PUBLIC TELEVISION (WEDH), KNPB Channel 5 (KNPB), SOPTV (KSYS), Rocky Mountain PBS (KRMA), KENW-TV3 (KENW), KUED Channel 7 (KUED), Wyoming PBS (KCWC), Colorado Public Television / KBDI 12 (KBDI), KBYU-TV (KBYU), Thirteen/WNET New York (WNET), WGBH/Channel 2 (WGBH), WGBY (WGBY), NJTV Public Media NJ (WNJT), WLIW21 (WLIW), mpt/Maryland Public Television (WMPB), WETA Television and Radio (WETA), WHYY (WHYY), PBS 39 (WLVT), WVPT - Your Source for PBS and More! (WVPT), Howard University Television (WHUT), WEDU PBS (WEDU), WGCU Public Media (WGCU), WPBT2 (WPBT), WUCF TV (WUCF), WUFT/Channel 5 (WUFT), WXEL/Channel 42 (WXEL), WLRN/Channel 17 (WLRN), WUSF Public Broadcasting (WUSF), ETV (WRLK), UNC-TV (WUNC), PBS Hawaii - Oceanic Cable Channel 10 (KHET), Idaho Public Television (KAID), KSPS (KSPS), OPB (KOPB), KWSU/Channel 10 & KTNW/Channel 31 (KWSU), WILL-TV (WILL), Network Knowledge - WSEC/Springfield (WSEC), WTTW11 (WTTW), Iowa Public Television/IPTV (KDIN), Nine Network (KETC), PBS39 Fort Wayne (WFWA), WFYI Indianapolis (WFYI), Milwaukee Public Television (WMVS), WNIN (WNIN), WNIT Public Television (WNIT), WPT (WPNE), WVUT/Channel 22 (WVUT), WEIU/Channel 51 (WEIU), WQPT-TV (WQPT), WYCC PBS Chicago (WYCC), WIPB-TV (WIPB), WTIU (WTIU), CET  (WCET), ThinkTVNetwork (WPTD), WBGU-TV (WBGU), WGVU TV (WGVU), NET1 (KUON), Pioneer Public Television (KWCM), SDPB Television (KUSD), TPT (KTCA), KSMQ (KSMQ), KPTS/Channel 8 (KPTS), KTWU/Channel 11 (KTWU), East Tennessee PBS (WSJK), WCTE-TV (WCTE), WLJT, Channel 11 (WLJT), WOSU TV (WOSU), WOUB/WOUC (WOUB), WVPB (WVPB), WKYU-PBS (WKYU), KERA 13 (KERA), MPBN (WCBB), Mountain Lake PBS (WCFE), NHPTV (WENH), Vermont PBS (WETK), witf (WITF), WQED Multimedia (WQED), WMHT Educational Telecommunications (WMHT), Q-TV (WDCQ), WTVS Detroit Public TV (WTVS), CMU Public Television (WCMU), WKAR-TV (WKAR), WNMU-TV Public TV 13 (WNMU), WDSE - WRPT (WDSE), WGTE TV (WGTE), Lakeland Public Television (KAWE), KMOS-TV - Channels 6.1, 6.2 and 6.3 (KMOS), MontanaPBS (KUSM), KRWG/Channel 22 (KRWG), KACV (KACV), KCOS/Channel 13 (KCOS), WCNY/Channel 24 (WCNY), WNED (WNED), WPBS (WPBS), WSKG Public TV (WSKG), WXXI (WXXI), WPSU (WPSU), WVIA Public Media Studios (WVIA), WTVI (WTVI), Western Reserve PBS (WNEO), WVIZ/PBS ideastream (WVIZ), KCTS 9 (KCTS), Basin PBS (KPBT), KUHT / Channel 8 (KUHT), KLRN (KLRN), KLRU (KLRU), WTJX Channel 12 (WTJX), WCVE PBS (WCVE), KBTC Public Television (KBTC)
@@ -628,6 +632,8 @@
 - **PhilharmonieDeParis**: Philharmonie de Paris
 - **phoenix.de**
 - **Photobucket**
+ - **Picarto**
+ - **PicartoVod**
 - **Piksel**
 - **Pinkbike**
 - **Pladform**
@@ -666,6 +672,8 @@
 - **qqmusic:playlist**: QQ音乐 - 歌单
 - **qqmusic:singer**: QQ音乐 - 歌手
 - **qqmusic:toplist**: QQ音乐 - 排行榜
+ - **Quickline**
+ - **QuicklineLive**
 - **R7**
 - **R7Article**
 - **radio.de**
@@ -783,7 +791,6 @@
 - **Spiegel**
 - **Spiegel:Article**: Articles on spiegel.de
 - **Spiegeltv**
- - **Spike**
 - **Sport5**
 - **SportBoxEmbed**
 - **SportDeutschland**
@@ -804,6 +811,7 @@
 - **SunPorno**
 - **SVT**
 - **SVTPlay**: SVT Play and Öppet arkiv
+ - **SVTSeries**
 - **SWRMediathek**
 - **Syfy**
 - **SztvHu**
@@ -1090,6 +1098,8 @@
 - **youtube:watchlater**: Youtube watch later list, ":ytwatchlater" for short (requires authentication)
 - **Zapiks**
 - **Zaq1**
+ - **Zattoo**
+ - **ZattooLive**
 - **ZDF**
 - **ZDFChannel**
 - **zingmp3**: mp3.zing.vn
--- a/test/test_subtitles.py
+++ b/test/test_subtitles.py
@@ -232,7 +232,7 @@ class TestNPOSubtitles(BaseTestSubtitles):


 class TestMTVSubtitles(BaseTestSubtitles):
-    url = 'http://www.cc.com/video-clips/kllhuv/stand-up-greg-fitzsimmons--uncensored---too-good-of-a-mother'
+    url = 'http://www.cc.com/video-clips/p63lk0/adam-devine-s-house-party-chasing-white-swans'
    IE = ComedyCentralIE

    def getInfoDict(self):
@@ -243,7 +243,7 @@ class TestMTVSubtitles(BaseTestSubtitles):
        self.DL.params['allsubtitles'] = True
        subtitles = self.getSubtitles()
        self.assertEqual(set(subtitles.keys()), set(['en']))
-        self.assertEqual(md5(subtitles['en']), 'b9f6ca22a6acf597ec76f61749765e65')
+        self.assertEqual(md5(subtitles['en']), '78206b8d8a0cfa9da64dc026eea48961')


 class TestNRKSubtitles(BaseTestSubtitles):
--- a/test/test_utils.py
+++ b/test/test_utils.py
@@ -42,6 +42,7 @@ from youtube_dl.utils import (
    is_html,
    js_to_json,
    limit_length,
+    merge_dicts,
    mimetype2ext,
    month_by_name,
    multipart_encode,
@@ -669,6 +670,17 @@ class TestUtil(unittest.TestCase):
            self.assertEqual(dict_get(d, ('b', 'c', key, )), None)
            self.assertEqual(dict_get(d, ('b', 'c', key, ), skip_false_values=False), false_value)

+    def test_merge_dicts(self):
+        self.assertEqual(merge_dicts({'a': 1}, {'b': 2}), {'a': 1, 'b': 2})
+        self.assertEqual(merge_dicts({'a': 1}, {'a': 2}), {'a': 1})
+        self.assertEqual(merge_dicts({'a': 1}, {'a': None}), {'a': 1})
+        self.assertEqual(merge_dicts({'a': 1}, {'a': ''}), {'a': 1})
+        self.assertEqual(merge_dicts({'a': 1}, {}), {'a': 1})
+        self.assertEqual(merge_dicts({'a': None}, {'a': 1}), {'a': 1})
+        self.assertEqual(merge_dicts({'a': ''}, {'a': 1}), {'a': ''})
+        self.assertEqual(merge_dicts({'a': ''}, {'a': 'abc'}), {'a': 'abc'})
+        self.assertEqual(merge_dicts({'a': None}, {'a': ''}, {'a': 'abc'}), {'a': 'abc'})
+
    def test_encode_compat_str(self):
        self.assertEqual(encode_compat_str(b'\xd1\x82\xd0\xb5\xd1\x81\xd1\x82', 'utf-8'), 'тест')
        self.assertEqual(encode_compat_str('тест', 'utf-8'), 'тест')
@@ -1072,6 +1084,18 @@ ffmpeg version 2.4.4 Copyright (c) 2000-2014 the FFmpeg ...'''), '2.4.4')
        self.assertFalse(match_str(
            'like_count > 100 & dislike_count <? 50 & description',
            {'like_count': 190, 'dislike_count': 10}))
+        self.assertTrue(match_str('is_live', {'is_live': True}))
+        self.assertFalse(match_str('is_live', {'is_live': False}))
+        self.assertFalse(match_str('is_live', {'is_live': None}))
+        self.assertFalse(match_str('is_live', {}))
+        self.assertFalse(match_str('!is_live', {'is_live': True}))
+        self.assertTrue(match_str('!is_live', {'is_live': False}))
+        self.assertTrue(match_str('!is_live', {'is_live': None}))
+        self.assertTrue(match_str('!is_live', {}))
+        self.assertTrue(match_str('title', {'title': 'abc'}))
+        self.assertTrue(match_str('title', {'title': ''}))
+        self.assertFalse(match_str('!title', {'title': 'abc'}))
+        self.assertFalse(match_str('!title', {'title': ''}))

    def test_parse_dfxp_time_expr(self):
        self.assertEqual(parse_dfxp_time_expr(None), None)
--- a/test/test_youtube_lists.py
+++ b/test/test_youtube_lists.py
@@ -61,7 +61,7 @@ class TestYoutubeLists(unittest.TestCase):
        dl = FakeYDL()
        dl.params['extract_flat'] = True
        ie = YoutubePlaylistIE(dl)
-        result = ie.extract('https://www.youtube.com/playlist?list=PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re')
+        result = ie.extract('https://www.youtube.com/playlist?list=PL-KKIb8rvtMSrAO9YFbeM6UQrAqoFTUWv')
        self.assertIsPlaylist(result)
        for entry in result['entries']:
            self.assertTrue(entry.get('title'))
--- a/youtube_dl/YoutubeDL.py
+++ b/youtube_dl/YoutubeDL.py
@@ -286,6 +286,9 @@ class YoutubeDL(object):
                       Two-letter ISO 3166-2 country code that will be used for
                       explicit geographic restriction bypassing via faking
                       X-Forwarded-For HTTP header (experimental)
+    geo_bypass_ip_block:
+                       IP range in CIDR notation that will be used similarly to
+                       geo_bypass_country (experimental)

    The following options determine which downloader is picked:
    external_downloader: Executable of the external downloader to call.
@@ -532,6 +535,8 @@ class YoutubeDL(object):
    def save_console_title(self):
        if not self.params.get('consoletitle', False):
            return
+        if self.params.get('simulate', False):
+            return
        if compat_os_name != 'nt' and 'TERM' in os.environ:
            # Save the title on stack
            self._write_string('\033[22;0t', self._screen_file)
@@ -539,6 +544,8 @@ class YoutubeDL(object):
    def restore_console_title(self):
        if not self.params.get('consoletitle', False):
            return
+        if self.params.get('simulate', False):
+            return
        if compat_os_name != 'nt' and 'TERM' in os.environ:
            # Restore the title from stack
            self._write_string('\033[23;0t', self._screen_file)
@@ -1475,23 +1482,28 @@ class YoutubeDL(object):
            if info_dict.get('%s_number' % field) is not None and not info_dict.get(field):
                info_dict[field] = '%s %d' % (field.capitalize(), info_dict['%s_number' % field])

+        for cc_kind in ('subtitles', 'automatic_captions'):
+            cc = info_dict.get(cc_kind)
+            if cc:
+                for _, subtitle in cc.items():
+                    for subtitle_format in subtitle:
+                        if subtitle_format.get('url'):
+                            subtitle_format['url'] = sanitize_url(subtitle_format['url'])
+                        if subtitle_format.get('ext') is None:
+                            subtitle_format['ext'] = determine_ext(subtitle_format['url']).lower()
+
+        automatic_captions = info_dict.get('automatic_captions')
        subtitles = info_dict.get('subtitles')
-        if subtitles:
-            for _, subtitle in subtitles.items():
-                for subtitle_format in subtitle:
-                    if subtitle_format.get('url'):
-                        subtitle_format['url'] = sanitize_url(subtitle_format['url'])
-                    if subtitle_format.get('ext') is None:
-                        subtitle_format['ext'] = determine_ext(subtitle_format['url']).lower()

        if self.params.get('listsubtitles', False):
            if 'automatic_captions' in info_dict:
-                self.list_subtitles(info_dict['id'], info_dict.get('automatic_captions'), 'automatic captions')
+                self.list_subtitles(
+                    info_dict['id'], automatic_captions, 'automatic captions')
            self.list_subtitles(info_dict['id'], subtitles, 'subtitles')
            return
+
        info_dict['requested_subtitles'] = self.process_subtitles(
-            info_dict['id'], subtitles,
-            info_dict.get('automatic_captions'))
+            info_dict['id'], subtitles, automatic_captions)

        # We now pick which formats have to be downloaded
        if info_dict.get('formats') is None:
@@ -1849,7 +1861,7 @@ class YoutubeDL(object):
                    def compatible_formats(formats):
                        video, audio = formats
                        # Check extension
-                        video_ext, audio_ext = audio.get('ext'), video.get('ext')
+                        video_ext, audio_ext = video.get('ext'), audio.get('ext')
                        if video_ext and audio_ext:
                            COMPATIBLE_EXTS = (
                                ('mp3', 'mp4', 'm4a', 'm4p', 'm4b', 'm4r', 'm4v', 'ismv', 'isma'),
--- a/youtube_dl/init.py
+++ b/youtube_dl/init.py
@@ -430,6 +430,7 @@ def _real_main(argv=None):
        'config_location': opts.config_location,
        'geo_bypass': opts.geo_bypass,
        'geo_bypass_country': opts.geo_bypass_country,
+        'geo_bypass_ip_block': opts.geo_bypass_ip_block,
        # just for deprecation check
        'autonumber': opts.autonumber if opts.autonumber is True else None,
        'usetitle': opts.usetitle if opts.usetitle is True else None,
--- a/youtube_dl/downloader/fragment.py
+++ b/youtube_dl/downloader/fragment.py
@@ -74,9 +74,14 @@ class FragmentFD(FileDownloader):
        return not ctx['live'] and not ctx['tmpfilename'] == '-'

    def _read_ytdl_file(self, ctx):
+        assert 'ytdl_corrupt' not in ctx
        stream, _ = sanitize_open(self.ytdl_filename(ctx['filename']), 'r')
-        ctx['fragment_index'] = json.loads(stream.read())['downloader']['current_fragment']['index']
-        stream.close()
+        try:
+            ctx['fragment_index'] = json.loads(stream.read())['downloader']['current_fragment']['index']
+        except Exception:
+            ctx['ytdl_corrupt'] = True
+        finally:
+            stream.close()

    def _write_ytdl_file(self, ctx):
        frag_index_stream, _ = sanitize_open(self.ytdl_filename(ctx['filename']), 'w')
@@ -158,11 +163,17 @@ class FragmentFD(FileDownloader):
        if self.__do_ytdl_file(ctx):
            if os.path.isfile(encodeFilename(self.ytdl_filename(ctx['filename']))):
                self._read_ytdl_file(ctx)
-                if ctx['fragment_index'] > 0 and resume_len == 0:
+                is_corrupt = ctx.get('ytdl_corrupt') is True
+                is_inconsistent = ctx['fragment_index'] > 0 and resume_len == 0
+                if is_corrupt or is_inconsistent:
+                    message = (
+                        '.ytdl file is corrupt' if is_corrupt else
+                        'Inconsistent state of incomplete fragment download')
                    self.report_warning(
-                        'Inconsistent state of incomplete fragment download. '
-                        'Restarting from the beginning...')
+                        '%s. Restarting from the beginning...' % message)
                    ctx['fragment_index'] = resume_len = 0
+                    if 'ytdl_corrupt' in ctx:
+                        del ctx['ytdl_corrupt']
                    self._write_ytdl_file(ctx)
            else:
                self._write_ytdl_file(ctx)
--- a/youtube_dl/extractor/acast.py
+++ b/youtube_dl/extractor/acast.py
@@ -7,7 +7,9 @@ import functools
 from .common import InfoExtractor
 from ..compat import compat_str
 from ..utils import (
+    float_or_none,
    int_or_none,
+    try_get,
    unified_timestamp,
    OnDemandPagedList,
 )
@@ -24,40 +26,58 @@ class ACastIE(InfoExtractor):
            'id': '57de3baa-4bb0-487e-9418-2692c1277a34',
            'ext': 'mp3',
            'title': '"Where Are You?": Taipei 101, Taiwan',
+            'description': 'md5:a0b4ef3634e63866b542e5b1199a1a0e',
            'timestamp': 1196172000,
            'upload_date': '20071127',
-            'description': 'md5:a0b4ef3634e63866b542e5b1199a1a0e',
            'duration': 211,
+            'creator': 'Concierge',
+            'series': 'Condé Nast Traveler Podcast',
+            'episode': '"Where Are You?": Taipei 101, Taiwan',
        }
    }, {
        # test with multiple blings
        'url': 'https://www.acast.com/sparpodcast/2.raggarmordet-rosterurdetforflutna',
-        'md5': 'e87d5b8516cd04c0d81b6ee1caca28d0',
+        'md5': 'a02393c74f3bdb1801c3ec2695577ce0',
        'info_dict': {
            'id': '2a92b283-1a75-4ad8-8396-499c641de0d9',
            'ext': 'mp3',
            'title': '2. Raggarmordet - Röster ur det förflutna',
+            'description': 'md5:4f81f6d8cf2e12ee21a321d8bca32db4',
            'timestamp': 1477346700,
            'upload_date': '20161024',
-            'description': 'md5:4f81f6d8cf2e12ee21a321d8bca32db4',
-            'duration': 2766,
+            'duration': 2766.602563,
+            'creator': 'Anton Berg & Martin Johnson',
+            'series': 'Spår',
+            'episode': '2. Raggarmordet - Röster ur det förflutna',
        }
    }]

    def _real_extract(self, url):
        channel, display_id = re.match(self._VALID_URL, url).groups()
+        s = self._download_json(
+            'https://play-api.acast.com/stitch/%s/%s' % (channel, display_id),
+            display_id)['result']
+        media_url = s['url']
        cast_data = self._download_json(
-            'https://play-api.acast.com/splash/%s/%s' % (channel, display_id), display_id)
-        e = cast_data['result']['episode']
+            'https://play-api.acast.com/splash/%s/%s' % (channel, display_id),
+            display_id)['result']
+        e = cast_data['episode']
+        title = e['name']
        return {
            'id': compat_str(e['id']),
            'display_id': display_id,
-            'url': e['mediaUrl'],
-            'title': e['name'],
-            'description': e.get('description'),
+            'url': media_url,
+            'title': title,
+            'description': e.get('description') or e.get('summary'),
            'thumbnail': e.get('image'),
            'timestamp': unified_timestamp(e.get('publishingDate')),
-            'duration': int_or_none(e.get('duration')),
+            'duration': float_or_none(s.get('duration') or e.get('duration')),
+            'filesize': int_or_none(e.get('contentLength')),
+            'creator': try_get(cast_data, lambda x: x['show']['author'], compat_str),
+            'series': try_get(cast_data, lambda x: x['show']['name'], compat_str),
+            'season_number': int_or_none(e.get('seasonNumber')),
+            'episode': title,
+            'episode_number': int_or_none(e.get('episodeNumber')),
        }


--- a/youtube_dl/extractor/americastestkitchen.py
+++ b/youtube_dl/extractor/americastestkitchen.py
--- a/youtube_dl/extractor/anvato.py
+++ b/youtube_dl/extractor/anvato.py
@@ -277,7 +277,9 @@ class AnvatoIE(InfoExtractor):

    def _real_extract(self, url):
        url, smuggled_data = unsmuggle_url(url, {})
-        self._initialize_geo_bypass(smuggled_data.get('geo_countries'))
+        self._initialize_geo_bypass({
+            'countries': smuggled_data.get('geo_countries'),
+        })

        mobj = re.match(self._VALID_URL, url)
        access_key, video_id = mobj.group('access_key_or_mcp', 'id')
--- a/youtube_dl/extractor/breakcom.py
+++ b/youtube_dl/extractor/breakcom.py
@@ -3,15 +3,13 @@ from __future__ import unicode_literals
 import re

 from .common import InfoExtractor
+from .youtube import YoutubeIE
 from ..compat import compat_str
-from ..utils import (
-    int_or_none,
-    parse_age_limit,
-)
+from ..utils import int_or_none


 class BreakIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?(?P<site>break|screenjunkies)\.com/video/(?P<display_id>[^/]+?)(?:-(?P<id>\d+))?(?:[/?#&]|$)'
+    _VALID_URL = r'https?://(?:www\.)?break\.com/video/(?P<display_id>[^/]+?)(?:-(?P<id>\d+))?(?:[/?#&]|$)'
    _TESTS = [{
        'url': 'http://www.break.com/video/when-girls-act-like-guys-2468056',
        'info_dict': {
@@ -19,125 +17,73 @@ class BreakIE(InfoExtractor):
            'ext': 'mp4',
            'title': 'When Girls Act Like D-Bags',
            'age_limit': 13,
+        },
+    }, {
+        # youtube embed
+        'url': 'http://www.break.com/video/someone-forgot-boat-brakes-work',
+        'info_dict': {
+            'id': 'RrrDLdeL2HQ',
+            'ext': 'mp4',
+            'title': 'Whale Watching Boat Crashing Into San Diego Dock',
+            'description': 'md5:afc1b2772f0a8468be51dd80eb021069',
+            'upload_date': '20160331',
+            'uploader': 'Steve Holden',
+            'uploader_id': 'sdholden07',
+        },
+        'params': {
+            'skip_download': True,
        }
-    }, {
-        'url': 'http://www.screenjunkies.com/video/best-quentin-tarantino-movie-2841915',
-        'md5': '5c2b686bec3d43de42bde9ec047536b0',
-        'info_dict': {
-            'id': '2841915',
-            'display_id': 'best-quentin-tarantino-movie',
-            'ext': 'mp4',
-            'title': 'Best Quentin Tarantino Movie',
-            'thumbnail': r're:^https?://.*\.jpg',
-            'duration': 3671,
-            'age_limit': 13,
-            'tags': list,
-        },
-    }, {
-        'url': 'http://www.screenjunkies.com/video/honest-trailers-the-dark-knight',
-        'info_dict': {
-            'id': '2348808',
-            'display_id': 'honest-trailers-the-dark-knight',
-            'ext': 'mp4',
-            'title': 'Honest Trailers - The Dark Knight',
-            'thumbnail': r're:^https?://.*\.(?:jpg|png)',
-            'age_limit': 10,
-            'tags': list,
-        },
-    }, {
-        # requires subscription but worked around
-        'url': 'http://www.screenjunkies.com/video/knocking-dead-ep-1-the-show-so-far-3003285',
-        'info_dict': {
-            'id': '3003285',
-            'display_id': 'knocking-dead-ep-1-the-show-so-far',
-            'ext': 'mp4',
-            'title': 'State of The Dead Recap: Knocking Dead Pilot',
-            'thumbnail': r're:^https?://.*\.jpg',
-            'duration': 3307,
-            'age_limit': 13,
-            'tags': list,
-        },
    }, {
        'url': 'http://www.break.com/video/ugc/baby-flex-2773063',
        'only_matching': True,
    }]

-    _DEFAULT_BITRATES = (48, 150, 320, 496, 864, 2240, 3264)
-
    def _real_extract(self, url):
-        site, display_id, video_id = re.match(self._VALID_URL, url).groups()
+        display_id, video_id = re.match(self._VALID_URL, url).groups()

-        if not video_id:
-            webpage = self._download_webpage(url, display_id)
-            video_id = self._search_regex(
-                (r'src=["\']/embed/(\d+)', r'data-video-content-id=["\'](\d+)'),
-                webpage, 'video id')
+        webpage = self._download_webpage(url, display_id)

-        webpage = self._download_webpage(
-            'http://www.%s.com/embed/%s' % (site, video_id),
-            display_id, 'Downloading video embed page')
-        embed_vars = self._parse_json(
+        youtube_url = YoutubeIE._extract_url(webpage)
+        if youtube_url:
+            return self.url_result(youtube_url, ie=YoutubeIE.ie_key())
+
+        content = self._parse_json(
            self._search_regex(
-                r'(?s)embedVars\s*=\s*({.+?})\s*</script>', webpage, 'embed vars'),
+                r'(?s)content["\']\s*:\s*(\[.+?\])\s*[,\n]', webpage,
+                'content'),
            display_id)

-        youtube_id = embed_vars.get('youtubeId')
-        if youtube_id:
-            return self.url_result(youtube_id, 'Youtube')
-
-        title = embed_vars['contentName']
-
        formats = []
-        bitrates = []
-        for f in embed_vars.get('media', []):
-            if not f.get('uri') or f.get('mediaPurpose') != 'play':
+        for video in content:
+            video_url = video.get('url')
+            if not video_url or not isinstance(video_url, compat_str):
                continue
-            bitrate = int_or_none(f.get('bitRate'))
-            if bitrate:
-                bitrates.append(bitrate)
+            bitrate = int_or_none(self._search_regex(
+                r'(\d+)_kbps', video_url, 'tbr', default=None))
            formats.append({
-                'url': f['uri'],
+                'url': video_url,
                'format_id': 'http-%d' % bitrate if bitrate else 'http',
-                'width': int_or_none(f.get('width')),
-                'height': int_or_none(f.get('height')),
                'tbr': bitrate,
-                'format': 'mp4',
            })
-
-        if not bitrates:
-            # When subscriptionLevel > 0, i.e. plus subscription is required
-            # media list will be empty. However, hds and hls uris are still
-            # available. We can grab them assuming bitrates to be default.
-            bitrates = self._DEFAULT_BITRATES
-
-        auth_token = embed_vars.get('AuthToken')
-
-        def construct_manifest_url(base_url, ext):
-            pieces = [base_url]
-            pieces.extend([compat_str(b) for b in bitrates])
-            pieces.append('_kbps.mp4.%s?%s' % (ext, auth_token))
-            return ','.join(pieces)
-
-        if bitrates and auth_token:
-            hds_url = embed_vars.get('hdsUri')
-            if hds_url:
-                formats.extend(self._extract_f4m_formats(
-                    construct_manifest_url(hds_url, 'f4m'),
-                    display_id, f4m_id='hds', fatal=False))
-            hls_url = embed_vars.get('hlsUri')
-            if hls_url:
-                formats.extend(self._extract_m3u8_formats(
-                    construct_manifest_url(hls_url, 'm3u8'),
-                    display_id, 'mp4', entry_protocol='m3u8_native', m3u8_id='hls', fatal=False))
        self._sort_formats(formats)

+        title = self._search_regex(
+            (r'title["\']\s*:\s*(["\'])(?P<value>(?:(?!\1).)+)\1',
+             r'<h1[^>]*>(?P<value>[^<]+)'), webpage, 'title', group='value')
+
+        def get(key, name):
+            return int_or_none(self._search_regex(
+                r'%s["\']\s*:\s*["\'](\d+)' % key, webpage, name,
+                default=None))
+
+        age_limit = get('ratings', 'age limit')
+        video_id = video_id or get('pid', 'video id') or display_id
+
        return {
            'id': video_id,
            'display_id': display_id,
            'title': title,
-            'thumbnail': embed_vars.get('thumbUri'),
-            'duration': int_or_none(embed_vars.get('videoLengthInSeconds')) or None,
-            'age_limit': parse_age_limit(embed_vars.get('audienceRating')),
-            'tags': embed_vars.get('tags', '').split(','),
+            'thumbnail': self._og_search_thumbnail(webpage),
+            'age_limit': age_limit,
            'formats': formats,
        }
--- a/youtube_dl/extractor/brightcove.py
+++ b/youtube_dl/extractor/brightcove.py
@@ -669,7 +669,10 @@ class BrightcoveNewIE(AdobePassIE):

    def _real_extract(self, url):
        url, smuggled_data = unsmuggle_url(url, {})
-        self._initialize_geo_bypass(smuggled_data.get('geo_countries'))
+        self._initialize_geo_bypass({
+            'countries': smuggled_data.get('geo_countries'),
+            'ip_blocks': smuggled_data.get('geo_ip_blocks'),
+        })

        account_id, player_id, embed, video_id = re.match(self._VALID_URL, url).groups()

--- a/youtube_dl/extractor/businessinsider.py
+++ b/youtube_dl/extractor/businessinsider.py
@@ -0,0 +1,42 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from .jwplatform import JWPlatformIE
+
+
+class BusinessInsiderIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:[^/]+\.)?businessinsider\.(?:com|nl)/(?:[^/]+/)*(?P<id>[^/?#&]+)'
+    _TESTS = [{
+        'url': 'http://uk.businessinsider.com/how-much-radiation-youre-exposed-to-in-everyday-life-2016-6',
+        'md5': 'ca237a53a8eb20b6dc5bd60564d4ab3e',
+        'info_dict': {
+            'id': 'hZRllCfw',
+            'ext': 'mp4',
+            'title': "Here's how much radiation you're exposed to in everyday life",
+            'description': 'md5:9a0d6e2c279948aadaa5e84d6d9b99bd',
+            'upload_date': '20170709',
+            'timestamp': 1499606400,
+        },
+        'params': {
+            'skip_download': True,
+        },
+    }, {
+        'url': 'https://www.businessinsider.nl/5-scientifically-proven-things-make-you-less-attractive-2017-7/',
+        'only_matching': True,
+    }, {
+        'url': 'http://www.businessinsider.com/excel-index-match-vlookup-video-how-to-2015-2?IR=T',
+        'only_matching': True,
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage = self._download_webpage(url, video_id)
+        jwplatform_id = self._search_regex(
+            (r'data-media-id=["\']([a-zA-Z0-9]{8})',
+             r'id=["\']jwplayer_([a-zA-Z0-9]{8})',
+             r'id["\']?\s*:\s*["\']?([a-zA-Z0-9]{8})'),
+            webpage, 'jwplatform id')
+        return self.url_result(
+            'jwplatform:%s' % jwplatform_id, ie=JWPlatformIE.ie_key(),
+            video_id=video_id)
--- a/youtube_dl/extractor/cbc.py
+++ b/youtube_dl/extractor/cbc.py
@@ -5,7 +5,10 @@ import json
 import re

 from .common import InfoExtractor
-from ..compat import compat_str
+from ..compat import (
+    compat_str,
+    compat_HTTPError,
+)
 from ..utils import (
    js_to_json,
    smuggle_url,
@@ -206,30 +209,48 @@ class CBCWatchBaseIE(InfoExtractor):

    def _call_api(self, path, video_id):
        url = path if path.startswith('http') else self._API_BASE_URL + path
-        result = self._download_xml(url, video_id, headers={
-            'X-Clearleap-DeviceId': self._device_id,
-            'X-Clearleap-DeviceToken': self._device_token,
-        })
+        for _ in range(2):
+            try:
+                result = self._download_xml(url, video_id, headers={
+                    'X-Clearleap-DeviceId': self._device_id,
+                    'X-Clearleap-DeviceToken': self._device_token,
+                })
+            except ExtractorError as e:
+                if isinstance(e.cause, compat_HTTPError) and e.cause.code == 401:
+                    # Device token has expired, re-acquiring device token
+                    self._register_device()
+                    continue
+                raise
        error_message = xpath_text(result, 'userMessage') or xpath_text(result, 'systemMessage')
        if error_message:
            raise ExtractorError('%s said: %s' % (self.IE_NAME, error_message))
        return result

    def _real_initialize(self):
-        if not self._device_id or not self._device_token:
-            device = self._downloader.cache.load('cbcwatch', 'device') or {}
-            self._device_id, self._device_token = device.get('id'), device.get('token')
-            if not self._device_id or not self._device_token:
-                result = self._download_xml(
-                    self._API_BASE_URL + 'device/register',
-                    None, data=b'<device><type>web</type></device>')
-                self._device_id = xpath_text(result, 'deviceId', fatal=True)
-                self._device_token = xpath_text(result, 'deviceToken', fatal=True)
-                self._downloader.cache.store(
-                    'cbcwatch', 'device', {
-                        'id': self._device_id,
-                        'token': self._device_token,
-                    })
+        if self._valid_device_token():
+            return
+        device = self._downloader.cache.load('cbcwatch', 'device') or {}
+        self._device_id, self._device_token = device.get('id'), device.get('token')
+        if self._valid_device_token():
+            return
+        self._register_device()
+
+    def _valid_device_token(self):
+        return self._device_id and self._device_token
+
+    def _register_device(self):
+        self._device_id = self._device_token = None
+        result = self._download_xml(
+            self._API_BASE_URL + 'device/register',
+            None, 'Acquiring device token',
+            data=b'<device><type>web</type></device>')
+        self._device_id = xpath_text(result, 'deviceId', fatal=True)
+        self._device_token = xpath_text(result, 'deviceToken', fatal=True)
+        self._downloader.cache.store(
+            'cbcwatch', 'device', {
+                'id': self._device_id,
+                'token': self._device_token,
+            })

    def _parse_rss_feed(self, rss):
        channel = xpath_element(rss, 'channel', fatal=True)
--- a/youtube_dl/extractor/cbs.py
+++ b/youtube_dl/extractor/cbs.py
@@ -65,7 +65,7 @@ class CBSIE(CBSBaseIE):
        last_e = None
        for item in items_data.findall('.//item'):
            asset_type = xpath_text(item, 'assetType')
-            if not asset_type or asset_type in asset_types:
+            if not asset_type or asset_type in asset_types or asset_type in ('HLS_FPS', 'DASH_CENC'):
                continue
            asset_types.append(asset_type)
            query = {
--- a/youtube_dl/extractor/cbssports.py
+++ b/youtube_dl/extractor/cbssports.py
@@ -4,28 +4,35 @@ from .cbs import CBSBaseIE


 class CBSSportsIE(CBSBaseIE):
-    _VALID_URL = r'https?://(?:www\.)?cbssports\.com/video/player/[^/]+/(?P<id>\d+)'
+    _VALID_URL = r'https?://(?:www\.)?cbssports\.com/[^/]+/(?:video|news)/(?P<id>[^/?#&]+)'

    _TESTS = [{
-        'url': 'http://www.cbssports.com/video/player/videos/708337219968/0/ben-simmons-the-next-lebron?-not-so-fast',
+        'url': 'https://www.cbssports.com/nba/video/donovan-mitchell-flashes-star-potential-in-game-2-victory-over-thunder/',
        'info_dict': {
-            'id': '708337219968',
+            'id': '1214315075735',
            'ext': 'mp4',
-            'title': 'Ben Simmons the next LeBron? Not so fast',
-            'description': 'md5:854294f627921baba1f4b9a990d87197',
-            'timestamp': 1466293740,
-            'upload_date': '20160618',
+            'title': 'Donovan Mitchell flashes star potential in Game 2 victory over Thunder',
+            'description': 'md5:df6f48622612c2d6bd2e295ddef58def',
+            'timestamp': 1524111457,
+            'upload_date': '20180419',
            'uploader': 'CBSI-NEW',
        },
        'params': {
            # m3u8 download
            'skip_download': True,
        }
+    }, {
+        'url': 'https://www.cbssports.com/nba/news/nba-playoffs-2018-watch-76ers-vs-heat-game-3-series-schedule-tv-channel-online-stream/',
+        'only_matching': True,
    }]

    def _extract_video_info(self, filter_query, video_id):
        return self._extract_feed_info('dJ5BDC', 'VxxJg8Ymh8sE', filter_query, video_id)

    def _real_extract(self, url):
-        video_id = self._match_id(url)
+        display_id = self._match_id(url)
+        webpage = self._download_webpage(url, display_id)
+        video_id = self._search_regex(
+            [r'(?:=|%26)pcid%3D(\d+)', r'embedVideo(?:Container)?_(\d+)'],
+            webpage, 'video id')
        return self._extract_video_info('byId=%s' % video_id, video_id)
--- a/youtube_dl/extractor/ccma.py
+++ b/youtube_dl/extractor/ccma.py
@@ -4,11 +4,13 @@ from __future__ import unicode_literals
 import re

 from .common import InfoExtractor
+from ..compat import compat_str
 from ..utils import (
+    clean_html,
    int_or_none,
    parse_duration,
    parse_iso8601,
-    clean_html,
+    parse_resolution,
 )


@@ -40,34 +42,42 @@ class CCMAIE(InfoExtractor):

    def _real_extract(self, url):
        media_type, media_id = re.match(self._VALID_URL, url).groups()
-        media_data = {}
-        formats = []
-        profiles = ['pc'] if media_type == 'audio' else ['mobil', 'pc']
-        for i, profile in enumerate(profiles):
-            md = self._download_json('http://dinamics.ccma.cat/pvideo/media.jsp', media_id, query={
+
+        media = self._download_json(
+            'http://dinamics.ccma.cat/pvideo/media.jsp', media_id, query={
                'media': media_type,
                'idint': media_id,
-                'profile': profile,
-            }, fatal=False)
-            if md:
-                media_data = md
-                media_url = media_data.get('media', {}).get('url')
-                if media_url:
-                    formats.append({
-                        'format_id': profile,
-                        'url': media_url,
-                        'quality': i,
-                    })
+            })
+
+        formats = []
+        media_url = media['media']['url']
+        if isinstance(media_url, list):
+            for format_ in media_url:
+                format_url = format_.get('file')
+                if not format_url or not isinstance(format_url, compat_str):
+                    continue
+                label = format_.get('label')
+                f = parse_resolution(label)
+                f.update({
+                    'url': format_url,
+                    'format_id': label,
+                })
+                formats.append(f)
+        else:
+            formats.append({
+                'url': media_url,
+                'vcodec': 'none' if media_type == 'audio' else None,
+            })
        self._sort_formats(formats)

-        informacio = media_data['informacio']
+        informacio = media['informacio']
        title = informacio['titol']
        durada = informacio.get('durada', {})
        duration = int_or_none(durada.get('milisegons'), 1000) or parse_duration(durada.get('text'))
        timestamp = parse_iso8601(informacio.get('data_emissio', {}).get('utc'))

        subtitles = {}
-        subtitols = media_data.get('subtitols', {})
+        subtitols = media.get('subtitols', {})
        if subtitols:
            sub_url = subtitols.get('url')
            if sub_url:
@@ -77,7 +87,7 @@ class CCMAIE(InfoExtractor):
                    })

        thumbnails = []
-        imatges = media_data.get('imatges', {})
+        imatges = media.get('imatges', {})
        if imatges:
            thumbnail_url = imatges.get('url')
            if thumbnail_url:
--- a/youtube_dl/extractor/cda.py
+++ b/youtube_dl/extractor/cda.py
--- a/youtube_dl/extractor/cloudflarestream.py
+++ b/youtube_dl/extractor/cloudflarestream.py
@@ -0,0 +1,60 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+import re
+
+from .common import InfoExtractor
+
+
+class CloudflareStreamIE(InfoExtractor):
+    _VALID_URL = r'''(?x)
+                    https?://
+                        (?:
+                            (?:watch\.)?cloudflarestream\.com/|
+                            embed\.cloudflarestream\.com/embed/[^/]+\.js\?.*?\bvideo=
+                        )
+                        (?P<id>[\da-f]+)
+                    '''
+    _TESTS = [{
+        'url': 'https://embed.cloudflarestream.com/embed/we4g.fla9.latest.js?video=31c9291ab41fac05471db4e73aa11717',
+        'info_dict': {
+            'id': '31c9291ab41fac05471db4e73aa11717',
+            'ext': 'mp4',
+            'title': '31c9291ab41fac05471db4e73aa11717',
+        },
+        'params': {
+            'skip_download': True,
+        },
+    }, {
+        'url': 'https://watch.cloudflarestream.com/9df17203414fd1db3e3ed74abbe936c1',
+        'only_matching': True,
+    }, {
+        'url': 'https://cloudflarestream.com/31c9291ab41fac05471db4e73aa11717/manifest/video.mpd',
+        'only_matching': True,
+    }]
+
+    @staticmethod
+    def _extract_urls(webpage):
+        return [
+            mobj.group('url')
+            for mobj in re.finditer(
+                r'<script[^>]+\bsrc=(["\'])(?P<url>(?:https?:)?//embed\.cloudflarestream\.com/embed/[^/]+\.js\?.*?\bvideo=[\da-f]+?.*?)\1',
+                webpage)]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+
+        formats = self._extract_m3u8_formats(
+            'https://cloudflarestream.com/%s/manifest/video.m3u8' % video_id,
+            video_id, 'mp4', entry_protocol='m3u8_native', m3u8_id='hls',
+            fatal=False)
+        formats.extend(self._extract_mpd_formats(
+            'https://cloudflarestream.com/%s/manifest/video.mpd' % video_id,
+            video_id, mpd_id='dash', fatal=False))
+        self._sort_formats(formats)
+
+        return {
+            'id': video_id,
+            'title': video_id,
+            'formats': formats,
+        }
--- a/youtube_dl/extractor/common.py
+++ b/youtube_dl/extractor/common.py
@@ -346,6 +346,11 @@ class InfoExtractor(object):
    geo restriction bypass mechanism right away in order to bypass
    geo restriction, of course, if the mechanism is not disabled. (experimental)

+    _GEO_IP_BLOCKS attribute may contain a list of presumably geo unrestricted
+    IP blocks in CIDR notation for this extractor. One of these IP blocks
+    will be used by geo restriction bypass mechanism similarly
+    to _GEO_COUNTRIES. (experimental)
+
    NB: both these geo attributes are experimental and may change in future
    or be completely removed.

@@ -358,6 +363,7 @@ class InfoExtractor(object):
    _x_forwarded_for_ip = None
    _GEO_BYPASS = True
    _GEO_COUNTRIES = None
+    _GEO_IP_BLOCKS = None
    _WORKING = True

    def __init__(self, downloader=None):
@@ -392,12 +398,15 @@ class InfoExtractor(object):

    def initialize(self):
        """Initializes an instance (authentication, etc)."""
-        self._initialize_geo_bypass(self._GEO_COUNTRIES)
+        self._initialize_geo_bypass({
+            'countries': self._GEO_COUNTRIES,
+            'ip_blocks': self._GEO_IP_BLOCKS,
+        })
        if not self._ready:
            self._real_initialize()
            self._ready = True

-    def _initialize_geo_bypass(self, countries):
+    def _initialize_geo_bypass(self, geo_bypass_context):
        """
        Initialize geo restriction bypass mechanism.

@@ -408,28 +417,82 @@ class InfoExtractor(object):
        HTTP requests.

        This method will be used for initial geo bypass mechanism initialization
-        during the instance initialization with _GEO_COUNTRIES.
+        during the instance initialization with _GEO_COUNTRIES and
+        _GEO_IP_BLOCKS.

-        You may also manually call it from extractor's code if geo countries
+        You may also manually call it from extractor's code if geo bypass
        information is not available beforehand (e.g. obtained during
-        extraction) or due to some another reason.
+        extraction) or due to some other reason. In this case you should pass
+        this information in geo bypass context passed as first argument. It may
+        contain following fields:
+
+        countries:  List of geo unrestricted countries (similar
+                    to _GEO_COUNTRIES)
+        ip_blocks:  List of geo unrestricted IP blocks in CIDR notation
+                    (similar to _GEO_IP_BLOCKS)
+
        """
        if not self._x_forwarded_for_ip:
-            country_code = self._downloader.params.get('geo_bypass_country', None)
-            # If there is no explicit country for geo bypass specified and
-            # the extractor is known to be geo restricted let's fake IP
-            # as X-Forwarded-For right away.
-            if (not country_code and
-                    self._GEO_BYPASS and
-                    self._downloader.params.get('geo_bypass', True) and
-                    countries):
-                country_code = random.choice(countries)
-            if country_code:
-                self._x_forwarded_for_ip = GeoUtils.random_ipv4(country_code)
+
+            # Geo bypass mechanism is explicitly disabled by user
+            if not self._downloader.params.get('geo_bypass', True):
+                return
+
+            if not geo_bypass_context:
+                geo_bypass_context = {}
+
+            # Backward compatibility: previously _initialize_geo_bypass
+            # expected a list of countries, some 3rd party code may still use
+            # it this way
+            if isinstance(geo_bypass_context, (list, tuple)):
+                geo_bypass_context = {
+                    'countries': geo_bypass_context,
+                }
+
+            # The whole point of geo bypass mechanism is to fake IP
+            # as X-Forwarded-For HTTP header based on some IP block or
+            # country code.
+
+            # Path 1: bypassing based on IP block in CIDR notation
+
+            # Explicit IP block specified by user, use it right away
+            # regardless of whether extractor is geo bypassable or not
+            ip_block = self._downloader.params.get('geo_bypass_ip_block', None)
+
+            # Otherwise use random IP block from geo bypass context but only
+            # if extractor is known as geo bypassable
+            if not ip_block:
+                ip_blocks = geo_bypass_context.get('ip_blocks')
+                if self._GEO_BYPASS and ip_blocks:
+                    ip_block = random.choice(ip_blocks)
+
+            if ip_block:
+                self._x_forwarded_for_ip = GeoUtils.random_ipv4(ip_block)
+                if self._downloader.params.get('verbose', False):
+                    self._downloader.to_screen(
+                        '[debug] Using fake IP %s as X-Forwarded-For.'
+                        % self._x_forwarded_for_ip)
+                return
+
+            # Path 2: bypassing based on country code
+
+            # Explicit country code specified by user, use it right away
+            # regardless of whether extractor is geo bypassable or not
+            country = self._downloader.params.get('geo_bypass_country', None)
+
+            # Otherwise use random country code from geo bypass context but
+            # only if extractor is known as geo bypassable
+            if not country:
+                countries = geo_bypass_context.get('countries')
+                if self._GEO_BYPASS and countries:
+                    country = random.choice(countries)
+
+            if country:
+                self._x_forwarded_for_ip = GeoUtils.random_ipv4(country)
                if self._downloader.params.get('verbose', False):
                    self._downloader.to_screen(
                        '[debug] Using fake IP %s (%s) as X-Forwarded-For.'
-                        % (self._x_forwarded_for_ip, country_code.upper()))
+                        % (self._x_forwarded_for_ip, country.upper()))

    def extract(self, url):
        """Extracts URL information and returns it in list of dicts."""
@@ -682,18 +745,30 @@ class InfoExtractor(object):
            else:
                self.report_warning(errmsg + str(ve))

-    def _download_json(self, url_or_request, video_id,
-                       note='Downloading JSON metadata',
-                       errnote='Unable to download JSON metadata',
-                       transform_source=None,
-                       fatal=True, encoding=None, data=None, headers={}, query={}):
-        json_string = self._download_webpage(
+    def _download_json_handle(
+            self, url_or_request, video_id, note='Downloading JSON metadata',
+            errnote='Unable to download JSON metadata', transform_source=None,
+            fatal=True, encoding=None, data=None, headers={}, query={}):
+        """Return a tuple (JSON object, URL handle)"""
+        res = self._download_webpage_handle(
            url_or_request, video_id, note, errnote, fatal=fatal,
            encoding=encoding, data=data, headers=headers, query=query)
-        if (not fatal) and json_string is False:
-            return None
+        if res is False:
+            return res
+        json_string, urlh = res
        return self._parse_json(
-            json_string, video_id, transform_source=transform_source, fatal=fatal)
+            json_string, video_id, transform_source=transform_source,
+            fatal=fatal), urlh
+
+    def _download_json(
+            self, url_or_request, video_id, note='Downloading JSON metadata',
+            errnote='Unable to download JSON metadata', transform_source=None,
+            fatal=True, encoding=None, data=None, headers={}, query={}):
+        res = self._download_json_handle(
+            url_or_request, video_id, note=note, errnote=errnote,
+            transform_source=transform_source, fatal=fatal, encoding=encoding,
+            data=data, headers=headers, query=query)
+        return res if res is False else res[0]

    def _parse_json(self, json_string, video_id, transform_source=None, fatal=True):
        if transform_source:
@@ -1008,6 +1083,40 @@ class InfoExtractor(object):
        if isinstance(json_ld, dict):
            json_ld = [json_ld]

+        INTERACTION_TYPE_MAP = {
+            'CommentAction': 'comment',
+            'AgreeAction': 'like',
+            'DisagreeAction': 'dislike',
+            'LikeAction': 'like',
+            'DislikeAction': 'dislike',
+            'ListenAction': 'view',
+            'WatchAction': 'view',
+            'ViewAction': 'view',
+        }
+
+        def extract_interaction_statistic(e):
+            interaction_statistic = e.get('interactionStatistic')
+            if not isinstance(interaction_statistic, list):
+                return
+            for is_e in interaction_statistic:
+                if not isinstance(is_e, dict):
+                    continue
+                if is_e.get('@type') != 'InteractionCounter':
+                    continue
+                interaction_type = is_e.get('interactionType')
+                if not isinstance(interaction_type, compat_str):
+                    continue
+                interaction_count = int_or_none(is_e.get('userInteractionCount'))
+                if interaction_count is None:
+                    continue
+                count_kind = INTERACTION_TYPE_MAP.get(interaction_type.split('/')[-1])
+                if not count_kind:
+                    continue
+                count_key = '%s_count' % count_kind
+                if info.get(count_key) is not None:
+                    continue
+                info[count_key] = interaction_count
+
        def extract_video_object(e):
            assert e['@type'] == 'VideoObject'
            info.update({
@@ -1023,9 +1132,10 @@ class InfoExtractor(object):
                'height': int_or_none(e.get('height')),
                'view_count': int_or_none(e.get('interactionCount')),
            })
+            extract_interaction_statistic(e)

        for e in json_ld:
-            if e.get('@context') == 'http://schema.org':
+            if isinstance(e.get('@context'), compat_str) and re.match(r'^https?://schema.org/?$', e.get('@context')):
                item_type = e.get('@type')
                if expected_type is not None and expected_type != item_type:
                    return info
--- a/youtube_dl/extractor/dailymotion.py
+++ b/youtube_dl/extractor/dailymotion.py
@@ -180,9 +180,12 @@ class DailymotionIE(DailymotionBaseInfoExtractor):
                        continue
                    ext = mimetype2ext(type_) or determine_ext(media_url)
                    if ext == 'm3u8':
-                        formats.extend(self._extract_m3u8_formats(
+                        m3u8_formats = self._extract_m3u8_formats(
                            media_url, video_id, 'mp4', preference=-1,
-                            m3u8_id='hls', fatal=False))
+                            m3u8_id='hls', fatal=False)
+                        for f in m3u8_formats:
+                            f['url'] = f['url'].split('#')[0]
+                            formats.append(f)
                    elif ext == 'f4m':
                        formats.extend(self._extract_f4m_formats(
                            media_url, video_id, preference=-1, f4m_id='hds', fatal=False))
--- a/youtube_dl/extractor/discovery.py
+++ b/youtube_dl/extractor/discovery.py
@@ -5,7 +5,10 @@ import re
 import string

 from .discoverygo import DiscoveryGoBaseIE
-from ..compat import compat_str
+from ..compat import (
+    compat_str,
+    compat_urllib_parse_unquote,
+)
 from ..utils import (
    ExtractorError,
    try_get,
@@ -55,15 +58,27 @@ class DiscoveryIE(DiscoveryGoBaseIE):
        video = next(cb for cb in content_blocks if cb.get('type') == 'video')['content']['items'][0]
        video_id = video['id']

-        access_token = self._download_json(
-            'https://www.%s.com/anonymous' % site, display_id, query={
-                'authRel': 'authorization',
-                'client_id': try_get(
-                    react_data, lambda x: x['application']['apiClientId'],
-                    compat_str) or '3020a40c2356a645b4b4',
-                'nonce': ''.join([random.choice(string.ascii_letters) for _ in range(32)]),
-                'redirectUri': 'https://fusion.ddmcdn.com/app/mercury-sdk/180/redirectHandler.html?https://www.%s.com' % site,
-            })['access_token']
+        access_token = None
+        cookies = self._get_cookies(url)
+
+        # prefer Affiliate Auth Token over Anonymous Auth Token
+        auth_storage_cookie = cookies.get('eosAf') or cookies.get('eosAn')
+        if auth_storage_cookie and auth_storage_cookie.value:
+            auth_storage = self._parse_json(compat_urllib_parse_unquote(
+                compat_urllib_parse_unquote(auth_storage_cookie.value)),
+                video_id, fatal=False) or {}
+            access_token = auth_storage.get('a') or auth_storage.get('access_token')
+
+        if not access_token:
+            access_token = self._download_json(
+                'https://www.%s.com/anonymous' % site, display_id, query={
+                    'authRel': 'authorization',
+                    'client_id': try_get(
+                        react_data, lambda x: x['application']['apiClientId'],
+                        compat_str) or '3020a40c2356a645b4b4',
+                    'nonce': ''.join([random.choice(string.ascii_letters) for _ in range(32)]),
+                    'redirectUri': 'https://fusion.ddmcdn.com/app/mercury-sdk/180/redirectHandler.html?https://www.%s.com' % site,
+                })['access_token']

        try:
            stream = self._download_json(
@@ -72,7 +87,7 @@ class DiscoveryIE(DiscoveryGoBaseIE):
                    'Authorization': 'Bearer ' + access_token,
                })
        except ExtractorError as e:
-            if isinstance(e.cause, compat_HTTPError) and e.cause.code == 403:
+            if isinstance(e.cause, compat_HTTPError) and e.cause.code in (401, 403):
                e_description = self._parse_json(
                    e.cause.read().decode(), display_id)['description']
                if 'resource not available for country' in e_description:
--- a/youtube_dl/extractor/dplay.py
+++ b/youtube_dl/extractor/dplay.py
@@ -102,7 +102,9 @@ class DPlayIE(InfoExtractor):
        display_id = mobj.group('id')
        domain = mobj.group('domain')

-        self._initialize_geo_bypass([mobj.group('country').upper()])
+        self._initialize_geo_bypass({
+            'countries': [mobj.group('country').upper()],
+        })

        webpage = self._download_webpage(url, display_id)

--- a/youtube_dl/extractor/dreisat.py
+++ b/youtube_dl/extractor/dreisat.py
@@ -8,7 +8,6 @@ from ..utils import (
    unified_strdate,
    xpath_text,
    determine_ext,
-    qualities,
    float_or_none,
    ExtractorError,
 )
@@ -16,7 +15,8 @@ from ..utils import (

 class DreiSatIE(InfoExtractor):
    IE_NAME = '3sat'
-    _VALID_URL = r'(?:https?://)?(?:www\.)?3sat\.de/mediathek/(?:index\.php|mediathek\.php)?\?(?:(?:mode|display)=[^&]+&)*obj=(?P<id>[0-9]+)$'
+    _GEO_COUNTRIES = ['DE']
+    _VALID_URL = r'https?://(?:www\.)?3sat\.de/mediathek/(?:(?:index|mediathek)\.php)?\?(?:(?:mode|display)=[^&]+&)*obj=(?P<id>[0-9]+)'
    _TESTS = [
        {
            'url': 'http://www.3sat.de/mediathek/index.php?mode=play&obj=45918',
@@ -43,7 +43,8 @@ class DreiSatIE(InfoExtractor):
    def _parse_smil_formats(self, smil, smil_url, video_id, namespace=None, f4m_params=None, transform_rtmp_url=None):
        param_groups = {}
        for param_group in smil.findall(self._xpath_ns('./head/paramGroup', namespace)):
-            group_id = param_group.attrib.get(self._xpath_ns('id', 'http://www.w3.org/XML/1998/namespace'))
+            group_id = param_group.get(self._xpath_ns(
+                'id', 'http://www.w3.org/XML/1998/namespace'))
            params = {}
            for param in param_group:
                params[param.get('name')] = param.get('value')
@@ -54,7 +55,7 @@ class DreiSatIE(InfoExtractor):
            src = video.get('src')
            if not src:
                continue
-            bitrate = float_or_none(video.get('system-bitrate') or video.get('systemBitrate'), 1000)
+            bitrate = int_or_none(self._search_regex(r'_(\d+)k', src, 'bitrate', None)) or float_or_none(video.get('system-bitrate') or video.get('systemBitrate'), 1000)
            group_id = video.get('paramGroup')
            param_group = param_groups[group_id]
            for proto in param_group['protocols'].split(','):
@@ -75,66 +76,36 @@ class DreiSatIE(InfoExtractor):
            note='Downloading video info',
            errnote='Failed to download video info')

-        status_code = doc.find('./status/statuscode')
-        if status_code is not None and status_code.text != 'ok':
-            code = status_code.text
-            if code == 'notVisibleAnymore':
+        status_code = xpath_text(doc, './status/statuscode')
+        if status_code and status_code != 'ok':
+            if status_code == 'notVisibleAnymore':
                message = 'Video %s is not available' % video_id
            else:
-                message = '%s returned error: %s' % (self.IE_NAME, code)
+                message = '%s returned error: %s' % (self.IE_NAME, status_code)
            raise ExtractorError(message, expected=True)

-        title = doc.find('.//information/title').text
-        description = xpath_text(doc, './/information/detail', 'description')
-        duration = int_or_none(xpath_text(doc, './/details/lengthSec', 'duration'))
-        uploader = xpath_text(doc, './/details/originChannelTitle', 'uploader')
-        uploader_id = xpath_text(doc, './/details/originChannelId', 'uploader id')
-        upload_date = unified_strdate(xpath_text(doc, './/details/airtime', 'upload date'))
+        title = xpath_text(doc, './/information/title', 'title', True)

-        def xml_to_thumbnails(fnode):
-            thumbnails = []
-            for node in fnode:
-                thumbnail_url = node.text
-                if not thumbnail_url:
-                    continue
-                thumbnail = {
-                    'url': thumbnail_url,
-                }
-                if 'key' in node.attrib:
-                    m = re.match('^([0-9]+)x([0-9]+)$', node.attrib['key'])
-                    if m:
-                        thumbnail['width'] = int(m.group(1))
-                        thumbnail['height'] = int(m.group(2))
-                thumbnails.append(thumbnail)
-            return thumbnails
-
-        thumbnails = xml_to_thumbnails(doc.findall('.//teaserimages/teaserimage'))
-
-        format_nodes = doc.findall('.//formitaeten/formitaet')
-        quality = qualities(['veryhigh', 'high', 'med', 'low'])
-
-        def get_quality(elem):
-            return quality(xpath_text(elem, 'quality'))
-        format_nodes.sort(key=get_quality)
-        format_ids = []
+        urls = []
        formats = []
-        for fnode in format_nodes:
-            video_url = fnode.find('url').text
-            is_available = 'http://www.metafilegenerator' not in video_url
-            if not is_available:
+        for fnode in doc.findall('.//formitaeten/formitaet'):
+            video_url = xpath_text(fnode, 'url')
+            if not video_url or video_url in urls:
                continue
+            urls.append(video_url)
+
+            is_available = 'http://www.metafilegenerator' not in video_url
+            geoloced = 'static_geoloced_online' in video_url
+            if not is_available or geoloced:
+                continue
+
            format_id = fnode.attrib['basetype']
-            quality = xpath_text(fnode, './quality', 'quality')
            format_m = re.match(r'''(?x)
                (?P<vcodec>[^_]+)_(?P<acodec>[^_]+)_(?P<container>[^_]+)_
                (?P<proto>[^_]+)_(?P<index>[^_]+)_(?P<indexproto>[^_]+)
            ''', format_id)

            ext = determine_ext(video_url, None) or format_m.group('container')
-            if ext not in ('smil', 'f4m', 'm3u8'):
-                format_id = format_id + '-' + quality
-            if format_id in format_ids:
-                continue

            if ext == 'meta':
                continue
@@ -147,24 +118,23 @@ class DreiSatIE(InfoExtractor):
                if video_url.startswith('https://'):
                    continue
                formats.extend(self._extract_m3u8_formats(
-                    video_url, video_id, 'mp4', m3u8_id=format_id, fatal=False))
+                    video_url, video_id, 'mp4', 'm3u8_native',
+                    m3u8_id=format_id, fatal=False))
            elif ext == 'f4m':
                formats.extend(self._extract_f4m_formats(
                    video_url, video_id, f4m_id=format_id, fatal=False))
            else:
-                proto = format_m.group('proto').lower()
+                quality = xpath_text(fnode, './quality')
+                if quality:
+                    format_id += '-' + quality

-                abr = int_or_none(xpath_text(fnode, './audioBitrate', 'abr'), 1000)
-                vbr = int_or_none(xpath_text(fnode, './videoBitrate', 'vbr'), 1000)
+                abr = int_or_none(xpath_text(fnode, './audioBitrate'), 1000)
+                vbr = int_or_none(xpath_text(fnode, './videoBitrate'), 1000)

-                width = int_or_none(xpath_text(fnode, './width', 'width'))
-                height = int_or_none(xpath_text(fnode, './height', 'height'))
-
-                filesize = int_or_none(xpath_text(fnode, './filesize', 'filesize'))
-
-                format_note = ''
-                if not format_note:
-                    format_note = None
+                tbr = int_or_none(self._search_regex(
+                    r'_(\d+)k', video_url, 'bitrate', None))
+                if tbr and vbr and not abr:
+                    abr = tbr - vbr

                formats.append({
                    'format_id': format_id,
@@ -174,31 +144,50 @@ class DreiSatIE(InfoExtractor):
                    'vcodec': format_m.group('vcodec'),
                    'abr': abr,
                    'vbr': vbr,
-                    'width': width,
-                    'height': height,
-                    'filesize': filesize,
-                    'format_note': format_note,
-                    'protocol': proto,
-                    '_available': is_available,
+                    'tbr': tbr,
+                    'width': int_or_none(xpath_text(fnode, './width')),
+                    'height': int_or_none(xpath_text(fnode, './height')),
+                    'filesize': int_or_none(xpath_text(fnode, './filesize')),
+                    'protocol': format_m.group('proto').lower(),
                })
-            format_ids.append(format_id)
+
+        geolocation = xpath_text(doc, './/details/geolocation')
+        if not formats and geolocation and geolocation != 'none':
+            self.raise_geo_restricted(countries=self._GEO_COUNTRIES)

        self._sort_formats(formats)

+        thumbnails = []
+        for node in doc.findall('.//teaserimages/teaserimage'):
+            thumbnail_url = node.text
+            if not thumbnail_url:
+                continue
+            thumbnail = {
+                'url': thumbnail_url,
+            }
+            thumbnail_key = node.get('key')
+            if thumbnail_key:
+                m = re.match('^([0-9]+)x([0-9]+)$', thumbnail_key)
+                if m:
+                    thumbnail['width'] = int(m.group(1))
+                    thumbnail['height'] = int(m.group(2))
+            thumbnails.append(thumbnail)
+
+        upload_date = unified_strdate(xpath_text(doc, './/details/airtime'))
+
        return {
            'id': video_id,
            'title': title,
-            'description': description,
-            'duration': duration,
+            'description': xpath_text(doc, './/information/detail'),
+            'duration': int_or_none(xpath_text(doc, './/details/lengthSec')),
            'thumbnails': thumbnails,
-            'uploader': uploader,
-            'uploader_id': uploader_id,
+            'uploader': xpath_text(doc, './/details/originChannelTitle'),
+            'uploader_id': xpath_text(doc, './/details/originChannelId'),
            'upload_date': upload_date,
            'formats': formats,
        }

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
-        video_id = mobj.group('id')
-        details_url = 'http://www.3sat.de/mediathek/xmlservice/web/beitragsDetails?ak=web&id=%s' % video_id
+        video_id = self._match_id(url)
+        details_url = 'http://www.3sat.de/mediathek/xmlservice/web/beitragsDetails?id=%s' % video_id
        return self.extract_from_xml_url(video_id, details_url)
--- a/youtube_dl/extractor/drtuber.py
+++ b/youtube_dl/extractor/drtuber.py
@@ -66,7 +66,9 @@ class DrTuberIE(InfoExtractor):
        self._sort_formats(formats)

        title = self._html_search_regex(
-            (r'class="title_watch"[^>]*><(?:p|h\d+)[^>]*>([^<]+)<',
+            (r'<h1[^>]+class=["\']title[^>]+>([^<]+)',
+             r'<title>([^<]+)\s*@\s+DrTuber',
+             r'class="title_watch"[^>]*><(?:p|h\d+)[^>]*>([^<]+)<',
             r'<p[^>]+class="title_substrate">([^<]+)</p>',
             r'<title>([^<]+) - \d+'),
            webpage, 'title')
--- a/youtube_dl/extractor/dtube.py
+++ b/youtube_dl/extractor/dtube.py
@@ -0,0 +1,83 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+import json
+import re
+from socket import timeout
+
+from .common import InfoExtractor
+from ..utils import (
+    int_or_none,
+    parse_iso8601,
+)
+
+
+class DTubeIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?d\.tube/(?:#!/)?v/(?P<uploader_id>[0-9a-z.-]+)/(?P<id>[0-9a-z]{8})'
+    _TEST = {
+        'url': 'https://d.tube/#!/v/benswann/zqd630em',
+        'md5': 'a03eaa186618ffa7a3145945543a251e',
+        'info_dict': {
+            'id': 'zqd630em',
+            'ext': 'mp4',
+            'title': 'Reality Check: FDA\'s Disinformation Campaign on Kratom',
+            'description': 'md5:700d164e066b87f9eac057949e4227c2',
+            'uploader_id': 'benswann',
+            'upload_date': '20180222',
+            'timestamp': 1519328958,
+        },
+        'params': {
+            'format': '480p',
+        },
+    }
+
+    def _real_extract(self, url):
+        uploader_id, video_id = re.match(self._VALID_URL, url).groups()
+        result = self._download_json('https://api.steemit.com/', video_id, data=json.dumps({
+            'jsonrpc': '2.0',
+            'method': 'get_content',
+            'params': [uploader_id, video_id],
+        }).encode())['result']
+
+        metadata = json.loads(result['json_metadata'])
+        video = metadata['video']
+        content = video['content']
+        info = video.get('info', {})
+        title = info.get('title') or result['title']
+
+        def canonical_url(h):
+            if not h:
+                return None
+            return 'https://ipfs.io/ipfs/' + h
+
+        formats = []
+        for q in ('240', '480', '720', '1080', ''):
+            video_url = canonical_url(content.get('video%shash' % q))
+            if not video_url:
+                continue
+            format_id = (q + 'p') if q else 'Source'
+            try:
+                self.to_screen('%s: Checking %s video format URL' % (video_id, format_id))
+                self._downloader._opener.open(video_url, timeout=5).close()
+            except timeout as e:
+                self.to_screen(
+                    '%s: %s URL is invalid, skipping' % (video_id, format_id))
+                continue
+            formats.append({
+                'format_id': format_id,
+                'url': video_url,
+                'height': int_or_none(q),
+                'ext': 'mp4',
+            })
+
+        return {
+            'id': video_id,
+            'title': title,
+            'description': content.get('description'),
+            'thumbnail': canonical_url(info.get('snaphash')),
+            'tags': content.get('tags') or metadata.get('tags'),
+            'duration': info.get('duration'),
+            'formats': formats,
+            'timestamp': parse_iso8601(result.get('created')),
+            'uploader_id': uploader_id,
+        }
--- a/youtube_dl/extractor/etonline.py
+++ b/youtube_dl/extractor/etonline.py
@@ -1,39 +0,0 @@
-# coding: utf-8
-from __future__ import unicode_literals
-
-import re
-
-from .common import InfoExtractor
-
-
-class ETOnlineIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?etonline\.com/(?:[^/]+/)*(?P<id>[^/?#&]+)'
-    _TESTS = [{
-        'url': 'http://www.etonline.com/tv/211130_dove_cameron_liv_and_maddie_emotional_episode_series_finale/',
-        'info_dict': {
-            'id': '211130_dove_cameron_liv_and_maddie_emotional_episode_series_finale',
-            'title': 'md5:a21ec7d3872ed98335cbd2a046f34ee6',
-            'description': 'md5:8b94484063f463cca709617c79618ccd',
-        },
-        'playlist_count': 2,
-    }, {
-        'url': 'http://www.etonline.com/media/video/here_are_the_stars_who_love_bringing_their_moms_as_dates_to_the_oscars-211359/',
-        'only_matching': True,
-    }]
-    BRIGHTCOVE_URL_TEMPLATE = 'http://players.brightcove.net/1242911076001/default_default/index.html?videoId=ref:%s'
-
-    def _real_extract(self, url):
-        playlist_id = self._match_id(url)
-
-        webpage = self._download_webpage(url, playlist_id)
-
-        entries = [
-            self.url_result(
-                self.BRIGHTCOVE_URL_TEMPLATE % video_id, 'BrightcoveNew', video_id)
-            for video_id in re.findall(
-                r'site\.brightcove\s*\([^,]+,\s*["\'](title_\d+)', webpage)]
-
-        return self.playlist_result(
-            entries, playlist_id,
-            self._og_search_title(webpage, fatal=False),
-            self._og_search_description(webpage))
--- a/youtube_dl/extractor/extractors.py
+++ b/youtube_dl/extractor/extractors.py
@@ -137,6 +137,7 @@ from .brightcove import (
    BrightcoveLegacyIE,
    BrightcoveNewIE,
 )
+from .businessinsider import BusinessInsiderIE
 from .buzzfeed import BuzzFeedIE
 from .byutv import BYUtvIE
 from .c56 import C56IE
@@ -195,6 +196,7 @@ from .clippit import ClippitIE
 from .cliprs import ClipRsIE
 from .clipsyndicate import ClipsyndicateIE
 from .closertotruth import CloserToTruthIE
+from .cloudflarestream import CloudflareStreamIE
 from .cloudy import CloudyIE
 from .clubic import ClubicIE
 from .clyp import ClypIE
@@ -281,6 +283,7 @@ from .drtv import (
    DRTVIE,
    DRTVLiveIE,
 )
+from .dtube import DTubeIE
 from .dvtv import DVTVIE
 from .dumpert import DumpertIE
 from .defense import DefenseGouvFrIE
@@ -326,7 +329,6 @@ from .espn import (
    FiveThirtyEightIE,
 )
 from .esri import EsriVideoIE
-from .etonline import ETOnlineIE
 from .europa import EuropaIE
 from .everyonesmixtape import EveryonesMixtapeIE
 from .expotv import ExpoTVIE
@@ -478,7 +480,10 @@ from .internetvideoarchive import InternetVideoArchiveIE
 from .iprima import IPrimaIE
 from .iqiyi import IqiyiIE
 from .ir90tv import Ir90TvIE
-from .itv import ITVIE
+from .itv import (
+    ITVIE,
+    ITVBTCCIE,
+)
 from .ivi import (
    IviIE,
    IviCompilationIE
@@ -577,7 +582,6 @@ from .mailru import (
    MailRuMusicIE,
    MailRuMusicSearchIE,
 )
-from .makerschannel import MakersChannelIE
 from .makertv import MakerTVIE
 from .mangomolo import (
    MangomoloVideoIE,
@@ -620,7 +624,6 @@ from .mnet import MnetIE
 from .moevideo import MoeVideoIE
 from .mofosex import MofosexIE
 from .mojvideo import MojvideoIE
-from .moniker import MonikerIE
 from .morningstar import MorningstarIE
 from .motherless import (
    MotherlessIE,
@@ -641,6 +644,7 @@ from .mtv import (
 from .muenchentv import MuenchenTVIE
 from .musicplayon import MusicPlayOnIE
 from .mwave import MwaveIE, MwaveMeetGreetIE
+from .mychannels import MyChannelsIE
 from .myspace import MySpaceIE, MySpaceAlbumIE
 from .myspass import MySpassIE
 from .myvi import (
@@ -815,6 +819,10 @@ from .periscope import (
 from .philharmoniedeparis import PhilharmonieDeParisIE
 from .phoenix import PhoenixIE
 from .photobucket import PhotobucketIE
+from .picarto import (
+    PicartoIE,
+    PicartoVodIE,
+)
 from .piksel import PikselIE
 from .pinkbike import PinkbikeIE
 from .pladform import PladformIE
@@ -1007,7 +1015,10 @@ from .spankbang import SpankBangIE
 from .spankwire import SpankwireIE
 from .spiegel import SpiegelIE, SpiegelArticleIE
 from .spiegeltv import SpiegeltvIE
-from .spike import SpikeIE
+from .spike import (
+    BellatorIE,
+    ParamountNetworkIE,
+)
 from .stitcher import StitcherIE
 from .sport5 import Sport5IE
 from .sportbox import SportBoxEmbedIE
@@ -1031,6 +1042,7 @@ from .sunporno import SunPornoIE
 from .svt import (
    SVTIE,
    SVTPlayIE,
+    SVTSeriesIE,
 )
 from .swrmediathek import SWRMediathekIE
 from .syfy import SyfyIE
@@ -1414,5 +1426,11 @@ from .youtube import (
 )
 from .zapiks import ZapiksIE
 from .zaq1 import Zaq1IE
+from .zattoo import (
+    QuicklineIE,
+    QuicklineLiveIE,
+    ZattooIE,
+    ZattooLiveIE,
+)
 from .zdf import ZDFIE, ZDFChannelIE
 from .zingmp3 import ZingMp3IE
--- a/youtube_dl/extractor/extremetube.py
+++ b/youtube_dl/extractor/extremetube.py
@@ -8,12 +8,12 @@ class ExtremeTubeIE(KeezMoviesIE):
    _VALID_URL = r'https?://(?:www\.)?extremetube\.com/(?:[^/]+/)?video/(?P<id>[^/#?&]+)'
    _TESTS = [{
        'url': 'http://www.extremetube.com/video/music-video-14-british-euro-brit-european-cumshots-swallow-652431',
-        'md5': '1fb9228f5e3332ec8c057d6ac36f33e0',
+        'md5': '92feaafa4b58e82f261e5419f39c60cb',
        'info_dict': {
            'id': 'music-video-14-british-euro-brit-european-cumshots-swallow-652431',
            'ext': 'mp4',
            'title': 'Music Video 14 british euro brit european cumshots swallow',
-            'uploader': 'unknown',
+            'uploader': 'anonim',
            'view_count': int,
            'age_limit': 18,
        }
@@ -36,10 +36,10 @@ class ExtremeTubeIE(KeezMoviesIE):
                r'<h1[^>]+title="([^"]+)"[^>]*>', webpage, 'title')

        uploader = self._html_search_regex(
-            r'Uploaded by:\s*</strong>\s*(.+?)\s*</div>',
+            r'Uploaded by:\s*</[^>]+>\s*<a[^>]+>(.+?)</a>',
            webpage, 'uploader', fatal=False)
        view_count = str_to_int(self._search_regex(
-            r'Views:\s*</strong>\s*<span>([\d,\.]+)</span>',
+            r'Views:\s*</[^>]+>\s*<[^>]+>([\d,\.]+)</',
            webpage, 'view count', fatal=False))

        info.update({
--- a/youtube_dl/extractor/funk.py
+++ b/youtube_dl/extractor/funk.py
@@ -5,7 +5,10 @@ import re

 from .common import InfoExtractor
 from .nexx import NexxIE
-from ..utils import int_or_none
+from ..utils import (
+    int_or_none,
+    try_get,
+)


 class FunkBaseIE(InfoExtractor):
@@ -77,6 +80,20 @@ class FunkChannelIE(FunkBaseIE):
        'params': {
            'skip_download': True,
        },
+    }, {
+        # only available via byIdList API
+        'url': 'https://www.funk.net/channel/informr/martin-sonneborn-erklaert-die-eu',
+        'info_dict': {
+            'id': '205067',
+            'ext': 'mp4',
+            'title': 'Martin Sonneborn erklärt die EU',
+            'description': 'md5:050f74626e4ed87edf4626d2024210c0',
+            'timestamp': 1494424042,
+            'upload_date': '20170510',
+        },
+        'params': {
+            'skip_download': True,
+        },
    }, {
        'url': 'https://www.funk.net/channel/59d5149841dca100012511e3/mein-erster-job-lovemilla-folge-1/lovemilla/',
        'only_matching': True,
@@ -87,16 +104,28 @@ class FunkChannelIE(FunkBaseIE):
        channel_id = mobj.group('id')
        alias = mobj.group('alias')

-        results = self._download_json(
-            'https://www.funk.net/api/v3.0/content/videos/filter', channel_id,
-            headers={
-                'authorization': 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjbGllbnROYW1lIjoiY3VyYXRpb24tdG9vbCIsInNjb3BlIjoic3RhdGljLWNvbnRlbnQtYXBpLGN1cmF0aW9uLWFwaSxzZWFyY2gtYXBpIn0.q4Y2xZG8PFHai24-4Pjx2gym9RmJejtmK6lMXP5wAgc',
-                'Referer': url,
-            }, query={
-                'channelId': channel_id,
-                'size': 100,
-            })['result']
+        headers = {
+            'authorization': 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjbGllbnROYW1lIjoiY3VyYXRpb24tdG9vbCIsInNjb3BlIjoic3RhdGljLWNvbnRlbnQtYXBpLGN1cmF0aW9uLWFwaSxzZWFyY2gtYXBpIn0.q4Y2xZG8PFHai24-4Pjx2gym9RmJejtmK6lMXP5wAgc',
+            'Referer': url,
+        }

-        video = next(r for r in results if r.get('alias') == alias)
+        video = None
+
+        by_id_list = self._download_json(
+            'https://www.funk.net/api/v3.0/content/videos/byIdList', channel_id,
+            headers=headers, query={
+                'ids': alias,
+            }, fatal=False)
+        if by_id_list:
+            video = try_get(by_id_list, lambda x: x['result'][0], dict)
+
+        if not video:
+            results = self._download_json(
+                'https://www.funk.net/api/v3.0/content/videos/filter', channel_id,
+                headers=headers, query={
+                    'channelId': channel_id,
+                    'size': 100,
+                })['result']
+            video = next(r for r in results if r.get('alias') == alias)

        return self._make_url_result(video)
--- a/youtube_dl/extractor/fxnetworks.py
+++ b/youtube_dl/extractor/fxnetworks.py
@@ -41,7 +41,7 @@ class FXNetworksIE(AdobePassIE):
        if 'The content you are trying to access is not available in your region.' in webpage:
            self.raise_geo_restricted()
        video_data = extract_attributes(self._search_regex(
-            r'(<a.+?rel="http://link\.theplatform\.com/s/.+?</a>)', webpage, 'video data'))
+            r'(<a.+?rel="https?://link\.theplatform\.com/s/.+?</a>)', webpage, 'video data'))
        player_type = self._search_regex(r'playerType\s*=\s*[\'"]([^\'"]+)', webpage, 'player type', default=None)
        release_url = video_data['rel']
        title = video_data['data-title']
--- a/youtube_dl/extractor/generic.py
+++ b/youtube_dl/extractor/generic.py
@@ -23,6 +23,7 @@ from ..utils import (
    is_html,
    js_to_json,
    KNOWN_EXTENSIONS,
+    merge_dicts,
    mimetype2ext,
    orderedSet,
    sanitized_Request,
@@ -58,6 +59,7 @@ from .xhamster import XHamsterEmbedIE
 from .tnaflix import TNAFlixNetworkEmbedIE
 from .drtuber import DrTuberIE
 from .redtube import RedTubeIE
+from .tube8 import Tube8IE
 from .vimeo import VimeoIE
 from .dailymotion import DailymotionIE
 from .dailymail import DailyMailIE
@@ -105,6 +107,7 @@ from .springboardplatform import SpringboardPlatformIE
 from .yapfiles import YapFilesIE
 from .vice import ViceIE
 from .xfileshare import XFileShareIE
+from .cloudflarestream import CloudflareStreamIE


 class GenericIE(InfoExtractor):
@@ -189,6 +192,16 @@ class GenericIE(InfoExtractor):
                'title': 'pdv_maddow_netcast_m4v-02-27-2015-201624',
            }
        },
+        # RSS feed with enclosures and unsupported link URLs
+        {
+            'url': 'http://www.hellointernet.fm/podcast?format=rss',
+            'info_dict': {
+                'id': 'http://www.hellointernet.fm/podcast?format=rss',
+                'description': 'CGP Grey and Brady Haran talk about YouTube, life, work, whatever.',
+                'title': 'Hello Internet',
+            },
+            'playlist_mincount': 100,
+        },
        # SMIL from http://videolectures.net/promogram_igor_mekjavic_eng
        {
            'url': 'http://videolectures.net/promogram_igor_mekjavic_eng/video/1/smil.xml',
@@ -1219,7 +1232,7 @@ class GenericIE(InfoExtractor):
                'title': '35871',
                'timestamp': 1355743100,
                'upload_date': '20121217',
-                'uploader_id': 'batchUser',
+                'uploader_id': 'cplapp@learn360.com',
            },
            'add_ie': ['Kaltura'],
        },
@@ -1270,6 +1283,39 @@ class GenericIE(InfoExtractor):
            },
            'add_ie': ['Kaltura'],
        },
+        {
+            # Kaltura iframe embed, more sophisticated
+            'url': 'http://www.cns.nyu.edu/~eero/math-tools/Videos/lecture-05sep2017.html',
+            'info_dict': {
+                'id': '1_9gzouybz',
+                'ext': 'mp4',
+                'title': 'lecture-05sep2017',
+                'description': 'md5:40f347d91fd4ba047e511c5321064b49',
+                'upload_date': '20170913',
+                'uploader_id': 'eps2',
+                'timestamp': 1505340777,
+            },
+            'params': {
+                'skip_download': True,
+            },
+            'add_ie': ['Kaltura'],
+        },
+        {
+            # meta twitter:player
+            'url': 'http://thechive.com/2017/12/08/all-i-want-for-christmas-is-more-twerk/',
+            'info_dict': {
+                'id': '0_01b42zps',
+                'ext': 'mp4',
+                'title': 'Main Twerk (Video)',
+                'upload_date': '20171208',
+                'uploader_id': 'sebastian.salinas@thechive.com',
+                'timestamp': 1512713057,
+            },
+            'params': {
+                'skip_download': True,
+            },
+            'add_ie': ['Kaltura'],
+        },
        # referrer protected EaglePlatform embed
        {
            'url': 'https://tvrain.ru/lite/teleshow/kak_vse_nachinalos/namin-418921/',
@@ -1426,21 +1472,6 @@ class GenericIE(InfoExtractor):
            },
            'expected_warnings': ['Failed to parse JSON Expecting value'],
        },
-        # Ooyala embed
-        {
-            'url': 'http://www.businessinsider.com/excel-index-match-vlookup-video-how-to-2015-2?IR=T',
-            'info_dict': {
-                'id': '50YnY4czr4ms1vJ7yz3xzq0excz_pUMs',
-                'ext': 'mp4',
-                'description': 'Index/Match versus VLOOKUP.',
-                'title': 'This is what separates the Excel masters from the wannabes',
-                'duration': 191.933,
-            },
-            'params': {
-                # m3u8 downloads
-                'skip_download': True,
-            }
-        },
        # Brightcove URL in single quotes
        {
            'url': 'http://www.sportsnet.ca/baseball/mlb/sn-presents-russell-martin-world-citizen/',
@@ -1967,7 +1998,30 @@ class GenericIE(InfoExtractor):
            'params': {
                'skip_download': True,
            },
-        }
+        },
+        {
+            # CloudflareStream embed
+            'url': 'https://www.cloudflare.com/products/cloudflare-stream/',
+            'info_dict': {
+                'id': '31c9291ab41fac05471db4e73aa11717',
+                'ext': 'mp4',
+                'title': '31c9291ab41fac05471db4e73aa11717',
+            },
+            'add_ie': [CloudflareStreamIE.ie_key()],
+            'params': {
+                'skip_download': True,
+            },
+        },
+        {
+            'url': 'http://share-videos.se/auto/video/83645793?uid=13',
+            'md5': 'b68d276de422ab07ee1d49388103f457',
+            'info_dict': {
+                'id': '83645793',
+                'title': 'Lock up and get excited',
+                'ext': 'mp4'
+            },
+            'skip': 'TODO: fix nested playlists processing in tests',
+        },
        # {
        #     # TODO: find another test
        #     # http://schema.org/VideoObject
@@ -1998,13 +2052,15 @@ class GenericIE(InfoExtractor):

        entries = []
        for it in doc.findall('./channel/item'):
-            next_url = xpath_text(it, 'link', fatal=False)
+            next_url = None
+            enclosure_nodes = it.findall('./enclosure')
+            for e in enclosure_nodes:
+                next_url = e.attrib.get('url')
+                if next_url:
+                    break
+
            if not next_url:
-                enclosure_nodes = it.findall('./enclosure')
-                for e in enclosure_nodes:
-                    next_url = e.attrib.get('url')
-                    if next_url:
-                        break
+                next_url = xpath_text(it, 'link', fatal=False)

            if not next_url:
                continue
@@ -2546,6 +2602,11 @@ class GenericIE(InfoExtractor):
        if redtube_urls:
            return self.playlist_from_matches(redtube_urls, video_id, video_title, ie=RedTubeIE.ie_key())

+        # Look for embedded Tube8 player
+        tube8_urls = Tube8IE._extract_urls(webpage)
+        if tube8_urls:
+            return self.playlist_from_matches(tube8_urls, video_id, video_title, ie=Tube8IE.ie_key())
+
        # Look for embedded Tvigle player
        mobj = re.search(
            r'<iframe[^>]+?src=(["\'])(?P<url>(?:https?:)?//cloud\.tvigle\.ru/video/.+?)\1', webpage)
@@ -2963,20 +3024,17 @@ class GenericIE(InfoExtractor):
            return self.playlist_from_matches(
                xfileshare_urls, video_id, video_title, ie=XFileShareIE.ie_key())

-        def merge_dicts(dict1, dict2):
-            merged = {}
-            for k, v in dict1.items():
-                if v is not None:
-                    merged[k] = v
-            for k, v in dict2.items():
-                if v is None:
-                    continue
-                if (k not in merged or
-                        (isinstance(v, compat_str) and v and
-                            isinstance(merged[k], compat_str) and
-                            not merged[k])):
-                    merged[k] = v
-            return merged
+        cloudflarestream_urls = CloudflareStreamIE._extract_urls(webpage)
+        if cloudflarestream_urls:
+            return self.playlist_from_matches(
+                cloudflarestream_urls, video_id, video_title, ie=CloudflareStreamIE.ie_key())
+
+        sharevideos_urls = [mobj.group('url') for mobj in re.finditer(
+            r'<iframe[^>]+?\bsrc\s*=\s*(["\'])(?P<url>(?:https?:)?//embed\.share-videos\.se/auto/embed/\d+\?.*?\buid=\d+.*?)\1',
+            webpage)]
+        if sharevideos_urls:
+            return self.playlist_from_matches(
+                sharevideos_urls, video_id, video_title)

        # Look for HTML5 media
        entries = self._parse_html5_media_entries(url, webpage, video_id, m3u8_id='hls')
--- a/youtube_dl/extractor/go.py
+++ b/youtube_dl/extractor/go.py
@@ -123,7 +123,7 @@ class GoIE(AdobePassIE):
                        'adobe_requestor_id': requestor_id,
                    })
                else:
-                    self._initialize_geo_bypass(['US'])
+                    self._initialize_geo_bypass({'countries': ['US']})
                entitlement = self._download_json(
                    'https://api.entitlement.watchabc.go.com/vp2/ws-secure/entitlement/2020/authorize.json',
                    video_id, data=urlencode_postdata(data))
--- a/youtube_dl/extractor/imdb.py
+++ b/youtube_dl/extractor/imdb.py
@@ -3,7 +3,9 @@ from __future__ import unicode_literals
 import re

 from .common import InfoExtractor
+from ..compat import compat_str
 from ..utils import (
+    determine_ext,
    mimetype2ext,
    qualities,
    remove_end,
@@ -73,19 +75,25 @@ class ImdbIE(InfoExtractor):
            video_info_list = format_info.get('videoInfoList')
            if not video_info_list or not isinstance(video_info_list, list):
                continue
-            video_info = video_info_list[0]
-            if not video_info or not isinstance(video_info, dict):
-                continue
-            video_url = video_info.get('videoUrl')
-            if not video_url:
-                continue
-            format_id = format_info.get('ffname')
-            formats.append({
-                'format_id': format_id,
-                'url': video_url,
-                'ext': mimetype2ext(video_info.get('videoMimeType')),
-                'quality': quality(format_id),
-            })
+            for video_info in video_info_list:
+                if not video_info or not isinstance(video_info, dict):
+                    continue
+                video_url = video_info.get('videoUrl')
+                if not video_url or not isinstance(video_url, compat_str):
+                    continue
+                if (video_info.get('videoMimeType') == 'application/x-mpegURL' or
+                        determine_ext(video_url) == 'm3u8'):
+                    formats.extend(self._extract_m3u8_formats(
+                        video_url, video_id, 'mp4', entry_protocol='m3u8_native',
+                        m3u8_id='hls', fatal=False))
+                    continue
+                format_id = format_info.get('ffname')
+                formats.append({
+                    'format_id': format_id,
+                    'url': video_url,
+                    'ext': mimetype2ext(video_info.get('videoMimeType')),
+                    'quality': quality(format_id),
+                })
        self._sort_formats(formats)

        return {
--- a/youtube_dl/extractor/instagram.py
+++ b/youtube_dl/extractor/instagram.py
@@ -1,15 +1,21 @@
 from __future__ import unicode_literals

 import itertools
+import hashlib
 import json
 import re

 from .common import InfoExtractor
-from ..compat import compat_str
+from ..compat import (
+    compat_str,
+    compat_HTTPError,
+)
 from ..utils import (
+    ExtractorError,
    get_element_by_attribute,
    int_or_none,
    lowercase_escape,
+    std_headers,
    try_get,
 )

@@ -238,23 +244,56 @@ class InstagramUserIE(InfoExtractor):
        }
    }

-    def _entries(self, uploader_id):
+    _gis_tmpl = None
+
+    def _entries(self, data):
        def get_count(suffix):
            return int_or_none(try_get(
                node, lambda x: x['edge_media_' + suffix]['count']))

+        uploader_id = data['entry_data']['ProfilePage'][0]['graphql']['user']['id']
+        csrf_token = data['config']['csrf_token']
+        rhx_gis = data.get('rhx_gis') or '3c7ca9dcefcf966d11dacf1f151335e8'
+
+        self._set_cookie('instagram.com', 'ig_pr', '1')
+
        cursor = ''
        for page_num in itertools.count(1):
-            media = self._download_json(
-                'https://www.instagram.com/graphql/query/', uploader_id,
-                'Downloading JSON page %d' % page_num, query={
-                    'query_hash': '472f257a40c653c64c666ce877d59d2b',
-                    'variables': json.dumps({
-                        'id': uploader_id,
-                        'first': 100,
-                        'after': cursor,
-                    })
-                })['data']['user']['edge_owner_to_timeline_media']
+            variables = json.dumps({
+                'id': uploader_id,
+                'first': 12,
+                'after': cursor,
+            })
+
+            if self._gis_tmpl:
+                gis_tmpls = [self._gis_tmpl]
+            else:
+                gis_tmpls = [
+                    '%s' % rhx_gis,
+                    '',
+                    '%s:%s' % (rhx_gis, csrf_token),
+                    '%s:%s:%s' % (rhx_gis, csrf_token, std_headers['User-Agent']),
+                ]
+
+            for gis_tmpl in gis_tmpls:
+                try:
+                    media = self._download_json(
+                        'https://www.instagram.com/graphql/query/', uploader_id,
+                        'Downloading JSON page %d' % page_num, headers={
+                            'X-Requested-With': 'XMLHttpRequest',
+                            'X-Instagram-GIS': hashlib.md5(
+                                ('%s:%s' % (gis_tmpl, variables)).encode('utf-8')).hexdigest(),
+                        }, query={
+                            'query_hash': '42323d64886122307be10013ad2dcc44',
+                            'variables': variables,
+                        })['data']['user']['edge_owner_to_timeline_media']
+                    self._gis_tmpl = gis_tmpl
+                    break
+                except ExtractorError as e:
+                    if isinstance(e.cause, compat_HTTPError) and e.cause.code == 403:
+                        if gis_tmpl != gis_tmpls[-1]:
+                            continue
+                    raise

            edges = media.get('edges')
            if not edges or not isinstance(edges, list):
@@ -309,9 +348,13 @@ class InstagramUserIE(InfoExtractor):

    def _real_extract(self, url):
        username = self._match_id(url)
-        uploader_id = self._download_json(
-            'https://instagram.com/%s/' % username, username, query={
-                '__a': 1,
-            })['graphql']['user']['id']
+
+        webpage = self._download_webpage(url, username)
+
+        data = self._parse_json(
+            self._search_regex(
+                r'sharedData\s*=\s*({.+?})\s*;\s*[<\n]', webpage, 'data'),
+            username)
+
        return self.playlist_result(
-            self._entries(uploader_id), username, username)
+            self._entries(data), username, username)
--- a/youtube_dl/extractor/itv.py
+++ b/youtube_dl/extractor/itv.py
@@ -7,6 +7,7 @@ import json
 import re

 from .common import InfoExtractor
+from .brightcove import BrightcoveNewIE
 from ..compat import (
    compat_str,
    compat_etree_register_namespace,
@@ -18,6 +19,7 @@ from ..utils import (
    xpath_text,
    int_or_none,
    parse_duration,
+    smuggle_url,
    ExtractorError,
    determine_ext,
 )
@@ -41,6 +43,14 @@ class ITVIE(InfoExtractor):
        # unavailable via data-playlist-url
        'url': 'https://www.itv.com/hub/through-the-keyhole/2a2271a0033',
        'only_matching': True,
+    }, {
+        # InvalidVodcrid
+        'url': 'https://www.itv.com/hub/james-martins-saturday-morning/2a5159a0034',
+        'only_matching': True,
+    }, {
+        # ContentUnavailable
+        'url': 'https://www.itv.com/hub/whos-doing-the-dishes/2a2898a0024',
+        'only_matching': True,
    }]

    def _real_extract(self, url):
@@ -127,7 +137,8 @@ class ITVIE(InfoExtractor):
            if fault_code == 'InvalidGeoRegion':
                self.raise_geo_restricted(
                    msg=fault_string, countries=self._GEO_COUNTRIES)
-            elif fault_code != 'InvalidEntity':
+            elif fault_code not in (
+                    'InvalidEntity', 'InvalidVodcrid', 'ContentUnavailable'):
                raise ExtractorError(
                    '%s said: %s' % (self.IE_NAME, fault_string), expected=True)
            info.update({
@@ -251,3 +262,38 @@ class ITVIE(InfoExtractor):
            'subtitles': subtitles,
        })
        return info
+
+
+class ITVBTCCIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?itv\.com/btcc/(?:[^/]+/)*(?P<id>[^/?#&]+)'
+    _TEST = {
+        'url': 'http://www.itv.com/btcc/races/btcc-2018-all-the-action-from-brands-hatch',
+        'info_dict': {
+            'id': 'btcc-2018-all-the-action-from-brands-hatch',
+            'title': 'BTCC 2018: All the action from Brands Hatch',
+        },
+        'playlist_mincount': 9,
+    }
+    BRIGHTCOVE_URL_TEMPLATE = 'http://players.brightcove.net/1582188683001/HkiHLnNRx_default/index.html?videoId=%s'
+
+    def _real_extract(self, url):
+        playlist_id = self._match_id(url)
+
+        webpage = self._download_webpage(url, playlist_id)
+
+        entries = [
+            self.url_result(
+                smuggle_url(self.BRIGHTCOVE_URL_TEMPLATE % video_id, {
+                    # ITV does not like some GB IP ranges, so here are some
+                    # IP blocks it accepts
+                    'geo_ip_blocks': [
+                        '193.113.0.0/16', '54.36.162.0/23', '159.65.16.0/21'
+                    ],
+                    'referrer': url,
+                }),
+                ie=BrightcoveNewIE.ie_key(), video_id=video_id)
+            for video_id in re.findall(r'data-video-id=["\'](\d+)', webpage)]
+
+        title = self._og_search_title(webpage, fatal=False)
+
+        return self.playlist_result(entries, playlist_id, title)
--- a/youtube_dl/extractor/joj.py
+++ b/youtube_dl/extractor/joj.py
--- a/youtube_dl/extractor/kaltura.py
+++ b/youtube_dl/extractor/kaltura.py
@@ -135,10 +135,11 @@ class KalturaIE(InfoExtractor):
                ''', webpage) or
            re.search(
                r'''(?xs)
-                    <iframe[^>]+src=(?P<q1>["'])
-                      (?:https?:)?//(?:www\.)?kaltura\.com/(?:(?!(?P=q1)).)*\b(?:p|partner_id)/(?P<partner_id>\d+)
+                    <(?:iframe[^>]+src|meta[^>]+\bcontent)=(?P<q1>["'])
+                      (?:https?:)?//(?:(?:www|cdnapi(?:sec)?)\.)?kaltura\.com/(?:(?!(?P=q1)).)*\b(?:p|partner_id)/(?P<partner_id>\d+)
+                      (?:(?!(?P=q1)).)*
+                      [?&;]entry_id=(?P<id>(?:(?!(?P=q1))[^&])+)
                      (?:(?!(?P=q1)).)*
-                      [?&]entry_id=(?P<id>(?:(?!(?P=q1))[^&])+)
                    (?P=q1)
                ''', webpage)
        )
--- a/youtube_dl/extractor/keezmovies.py
+++ b/youtube_dl/extractor/keezmovies.py
@@ -20,23 +20,23 @@ from ..utils import (
 class KeezMoviesIE(InfoExtractor):
    _VALID_URL = r'https?://(?:www\.)?keezmovies\.com/video/(?:(?P<display_id>[^/]+)-)?(?P<id>\d+)'
    _TESTS = [{
-        'url': 'http://www.keezmovies.com/video/petite-asian-lady-mai-playing-in-bathtub-1214711',
-        'md5': '1c1e75d22ffa53320f45eeb07bc4cdc0',
+        'url': 'https://www.keezmovies.com/video/arab-wife-want-it-so-bad-i-see-she-thirsty-and-has-tiny-money-18070681',
+        'md5': '2ac69cdb882055f71d82db4311732a1a',
        'info_dict': {
-            'id': '1214711',
-            'display_id': 'petite-asian-lady-mai-playing-in-bathtub',
+            'id': '18070681',
+            'display_id': 'arab-wife-want-it-so-bad-i-see-she-thirsty-and-has-tiny-money',
            'ext': 'mp4',
-            'title': 'Petite Asian Lady Mai Playing In Bathtub',
-            'thumbnail': r're:^https?://.*\.jpg$',
+            'title': 'Arab wife want it so bad I see she thirsty and has tiny money.',
+            'thumbnail': None,
            'view_count': int,
            'age_limit': 18,
        }
    }, {
-        'url': 'http://www.keezmovies.com/video/1214711',
+        'url': 'http://www.keezmovies.com/video/18070681',
        'only_matching': True,
    }]

-    def _extract_info(self, url):
+    def _extract_info(self, url, fatal=True):
        mobj = re.match(self._VALID_URL, url)
        video_id = mobj.group('id')
        display_id = (mobj.group('display_id')
@@ -55,7 +55,7 @@ class KeezMoviesIE(InfoExtractor):
        encrypted = False

        def extract_format(format_url, height=None):
-            if not isinstance(format_url, compat_str) or not format_url.startswith('http'):
+            if not isinstance(format_url, compat_str) or not format_url.startswith(('http', '//')):
                return
            if format_url in format_urls:
                return
@@ -105,7 +105,11 @@ class KeezMoviesIE(InfoExtractor):
                raise ExtractorError(
                    'Video %s is no longer available' % video_id, expected=True)

-        self._sort_formats(formats)
+        try:
+            self._sort_formats(formats)
+        except ExtractorError:
+            if fatal:
+                raise

        if not title:
            title = self._html_search_regex(
@@ -122,7 +126,9 @@ class KeezMoviesIE(InfoExtractor):
        }

    def _real_extract(self, url):
-        webpage, info = self._extract_info(url)
+        webpage, info = self._extract_info(url, fatal=False)
+        if not info['formats']:
+            return self.url_result(url, 'Generic')
        info['view_count'] = str_to_int(self._search_regex(
            r'<b>([\d,.]+)</b> Views?', webpage, 'view count', fatal=False))
        return info
--- a/youtube_dl/extractor/limelight.py
+++ b/youtube_dl/extractor/limelight.py
@@ -282,7 +282,9 @@ class LimelightMediaIE(LimelightBaseIE):
    def _real_extract(self, url):
        url, smuggled_data = unsmuggle_url(url, {})
        video_id = self._match_id(url)
-        self._initialize_geo_bypass(smuggled_data.get('geo_countries'))
+        self._initialize_geo_bypass({
+            'countries': smuggled_data.get('geo_countries'),
+        })

        pc, mobile, metadata = self._extract(
            video_id, 'getPlaylistByMediaId',
--- a/youtube_dl/extractor/liveleak.py
+++ b/youtube_dl/extractor/liveleak.py
@@ -7,7 +7,7 @@ from ..utils import int_or_none


 class LiveLeakIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:\w+\.)?liveleak\.com/view\?(?:.*?)i=(?P<id>[\w_]+)(?:.*)'
+    _VALID_URL = r'https?://(?:\w+\.)?liveleak\.com/view\?.*?\b[it]=(?P<id>[\w_]+)'
    _TESTS = [{
        'url': 'http://www.liveleak.com/view?i=757_1364311680',
        'md5': '0813c2430bea7a46bf13acf3406992f4',
@@ -79,6 +79,9 @@ class LiveLeakIE(InfoExtractor):
            'title': 'Fuel Depot in China Explosion caught on video',
        },
        'playlist_count': 3,
+    }, {
+        'url': 'https://www.liveleak.com/view?t=HvHi_1523016227',
+        'only_matching': True,
    }]

    @staticmethod
--- a/youtube_dl/extractor/minoto.py
+++ b/youtube_dl/extractor/minoto.py
@@ -4,7 +4,10 @@ from __future__ import unicode_literals
 import re

 from .common import InfoExtractor
-from ..utils import int_or_none
+from ..utils import (
+    int_or_none,
+    parse_codecs,
+)


 class MinotoIE(InfoExtractor):
@@ -26,7 +29,7 @@ class MinotoIE(InfoExtractor):
                formats.extend(fmt_url, video_id, 'mp4', m3u8_id='hls', fatal=False)
            else:
                fmt_profile = fmt.get('profile') or {}
-                f = {
+                formats.append({
                    'format_id': fmt_profile.get('name-short'),
                    'format_note': fmt_profile.get('name'),
                    'url': fmt_url,
@@ -35,16 +38,8 @@ class MinotoIE(InfoExtractor):
                    'filesize': int_or_none(fmt.get('filesize')),
                    'width': int_or_none(fmt.get('width')),
                    'height': int_or_none(fmt.get('height')),
-                }
-                codecs = fmt.get('codecs')
-                if codecs:
-                    codecs = codecs.split(',')
-                    if len(codecs) == 2:
-                        f.update({
-                            'vcodec': codecs[0],
-                            'acodec': codecs[1],
-                        })
-                formats.append(f)
+                    'codecs': parse_codecs(fmt.get('codecs')),
+                })
        self._sort_formats(formats)

        return {
--- a/youtube_dl/extractor/mixcloud.py
+++ b/youtube_dl/extractor/mixcloud.py
@@ -179,6 +179,10 @@ class MixcloudIE(InfoExtractor):
                    formats.append({
                        'format_id': 'http',
                        'url': decrypted,
+                        'downloader_options': {
+                            # Mixcloud starts throttling at >~5M
+                            'http_chunk_size': 5242880,
+                        },
                    })
            self._sort_formats(formats)

--- a/youtube_dl/extractor/mofosex.py
+++ b/youtube_dl/extractor/mofosex.py
@@ -12,7 +12,7 @@ class MofosexIE(KeezMoviesIE):
    _VALID_URL = r'https?://(?:www\.)?mofosex\.com/videos/(?P<id>\d+)/(?P<display_id>[^/?#&.]+)\.html'
    _TESTS = [{
        'url': 'http://www.mofosex.com/videos/318131/amateur-teen-playing-and-masturbating-318131.html',
-        'md5': '39a15853632b7b2e5679f92f69b78e91',
+        'md5': '558fcdafbb63a87c019218d6e49daf8a',
        'info_dict': {
            'id': '318131',
            'display_id': 'amateur-teen-playing-and-masturbating-318131',
--- a/youtube_dl/extractor/moniker.py
+++ b/youtube_dl/extractor/moniker.py
@@ -1,116 +0,0 @@
-# coding: utf-8
-from __future__ import unicode_literals
-
-import os.path
-import re
-
-from .common import InfoExtractor
-from ..utils import (
-    ExtractorError,
-    remove_start,
-    sanitized_Request,
-    urlencode_postdata,
-)
-
-
-class MonikerIE(InfoExtractor):
-    IE_DESC = 'allmyvideos.net and vidspot.net'
-    _VALID_URL = r'https?://(?:www\.)?(?:allmyvideos|vidspot)\.net/(?:(?:2|v)/v-)?(?P<id>[a-zA-Z0-9_-]+)'
-
-    _TESTS = [{
-        'url': 'http://allmyvideos.net/jih3nce3x6wn',
-        'md5': '710883dee1bfc370ecf9fa6a89307c88',
-        'info_dict': {
-            'id': 'jih3nce3x6wn',
-            'ext': 'mp4',
-            'title': 'youtube-dl test video',
-        },
-    }, {
-        'url': 'http://allmyvideos.net/embed-jih3nce3x6wn',
-        'md5': '710883dee1bfc370ecf9fa6a89307c88',
-        'info_dict': {
-            'id': 'jih3nce3x6wn',
-            'ext': 'mp4',
-            'title': 'youtube-dl test video',
-        },
-    }, {
-        'url': 'http://vidspot.net/l2ngsmhs8ci5',
-        'md5': '710883dee1bfc370ecf9fa6a89307c88',
-        'info_dict': {
-            'id': 'l2ngsmhs8ci5',
-            'ext': 'mp4',
-            'title': 'youtube-dl test video',
-        },
-    }, {
-        'url': 'https://www.vidspot.net/l2ngsmhs8ci5',
-        'only_matching': True,
-    }, {
-        'url': 'http://vidspot.net/2/v-ywDf99',
-        'md5': '5f8254ce12df30479428b0152fb8e7ba',
-        'info_dict': {
-            'id': 'ywDf99',
-            'ext': 'mp4',
-            'title': 'IL FAIT LE MALIN EN PORSHE CAYENNE ( mais pas pour longtemps)',
-            'description': 'IL FAIT LE MALIN EN PORSHE CAYENNE.',
-        },
-    }, {
-        'url': 'http://allmyvideos.net/v/v-HXZm5t',
-        'only_matching': True,
-    }]
-
-    def _real_extract(self, url):
-        orig_video_id = self._match_id(url)
-        video_id = remove_start(orig_video_id, 'embed-')
-        url = url.replace(orig_video_id, video_id)
-        assert re.match(self._VALID_URL, url) is not None
-        orig_webpage = self._download_webpage(url, video_id)
-
-        if '>File Not Found<' in orig_webpage:
-            raise ExtractorError('Video %s does not exist' % video_id, expected=True)
-
-        error = self._search_regex(
-            r'class="err">([^<]+)<', orig_webpage, 'error', default=None)
-        if error:
-            raise ExtractorError(
-                '%s returned error: %s' % (self.IE_NAME, error), expected=True)
-
-        builtin_url = self._search_regex(
-            r'<iframe[^>]+src=(["\'])(?P<url>.+?/builtin-.+?)\1',
-            orig_webpage, 'builtin URL', default=None, group='url')
-
-        if builtin_url:
-            req = sanitized_Request(builtin_url)
-            req.add_header('Referer', url)
-            webpage = self._download_webpage(req, video_id, 'Downloading builtin page')
-            title = self._og_search_title(orig_webpage).strip()
-            description = self._og_search_description(orig_webpage).strip()
-        else:
-            fields = re.findall(r'type="hidden" name="(.+?)"\s* value="?(.+?)">', orig_webpage)
-            data = dict(fields)
-
-            post = urlencode_postdata(data)
-            headers = {
-                b'Content-Type': b'application/x-www-form-urlencoded',
-            }
-            req = sanitized_Request(url, post, headers)
-            webpage = self._download_webpage(
-                req, video_id, note='Downloading video page ...')
-
-            title = os.path.splitext(data['fname'])[0]
-            description = None
-
-        # Could be several links with different quality
-        links = re.findall(r'"file" : "?(.+?)",', webpage)
-        # Assume the links are ordered in quality
-        formats = [{
-            'url': l,
-            'quality': i,
-        } for i, l in enumerate(links)]
-        self._sort_formats(formats)
-
-        return {
-            'id': video_id,
-            'title': title,
-            'description': description,
-            'formats': formats,
-        }
--- a/youtube_dl/extractor/makerschannel.py
+++ b/youtube_dl/extractor/makerschannel.py
@@ -6,17 +6,17 @@ import re
 from .common import InfoExtractor


-class MakersChannelIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?makerschannel\.com/.*(?P<id_type>video|production)_id=(?P<id>[0-9]+)'
+class MyChannelsIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?mychannels\.com/.*(?P<id_type>video|production)_id=(?P<id>[0-9]+)'
    _TEST = {
-        'url': 'http://makerschannel.com/en/zoomin/community-highlights?video_id=849',
-        'md5': '624a512c6969236b5967bf9286345ad1',
+        'url': 'https://mychannels.com/missholland/miss-holland?production_id=3416',
+        'md5': 'b8993daad4262dd68d89d651c0c52c45',
        'info_dict': {
-            'id': '849',
+            'id': 'wUUDZZep6vQD',
            'ext': 'mp4',
-            'title': 'Landing a bus on a plane is an epic win',
-            'uploader': 'ZoomIn',
-            'description': 'md5:cd9cca2ea7b69b78be81d07020c97139',
+            'title': 'Miss Holland joins VOTE LEAVE',
+            'description': 'Miss Holland | #13 Not a potato',
+            'uploader': 'Miss Holland',
        }
    }

@@ -27,12 +27,12 @@ class MakersChannelIE(InfoExtractor):

        def extract_data_val(attr, fatal=False):
            return self._html_search_regex(r'data-%s\s*=\s*"([^"]+)"' % attr, video_data, attr, fatal=fatal)
-        minoto_id = self._search_regex(r'/id/([a-zA-Z0-9]+)', extract_data_val('video-src', True), 'minoto id')
+        minoto_id = extract_data_val('minoto-id') or self._search_regex(r'/id/([a-zA-Z0-9]+)', extract_data_val('video-src', True), 'minoto id')

        return {
            '_type': 'url_transparent',
            'url': 'minoto:%s' % minoto_id,
-            'id': extract_data_val('video-id', True),
+            'id': url_id,
            'title': extract_data_val('title', True),
            'description': extract_data_val('description'),
            'thumbnail': extract_data_val('image'),
--- a/youtube_dl/extractor/nbc.py
+++ b/youtube_dl/extractor/nbc.py
@@ -9,6 +9,7 @@ from .adobepass import AdobePassIE
 from ..utils import (
    find_xpath_attr,
    smuggle_url,
+    try_get,
    unescapeHTML,
    update_url_query,
    int_or_none,
@@ -78,10 +79,14 @@ class NBCIE(AdobePassIE):
    def _real_extract(self, url):
        permalink, video_id = re.match(self._VALID_URL, url).groups()
        permalink = 'http' + permalink
-        video_data = self._download_json(
+        response = self._download_json(
            'https://api.nbc.com/v3/videos', video_id, query={
                'filter[permalink]': permalink,
-            })['data'][0]['attributes']
+                'fields[videos]': 'description,entitlement,episodeNumber,guid,keywords,seasonNumber,title,vChipRating',
+                'fields[shows]': 'shortTitle',
+                'include': 'show.shortTitle',
+            })
+        video_data = response['data'][0]['attributes']
        query = {
            'mbr': 'true',
            'manifest': 'm3u',
@@ -103,10 +108,11 @@ class NBCIE(AdobePassIE):
            'title': title,
            'url': theplatform_url,
            'description': video_data.get('description'),
-            'keywords': video_data.get('keywords'),
+            'tags': video_data.get('keywords'),
            'season_number': int_or_none(video_data.get('seasonNumber')),
            'episode_number': int_or_none(video_data.get('episodeNumber')),
-            'series': video_data.get('showName'),
+            'episode': title,
+            'series': try_get(response, lambda x: x['included'][0]['attributes']['shortTitle']),
            'ie_key': 'ThePlatform',
        }

--- a/youtube_dl/extractor/nexx.py
+++ b/youtube_dl/extractor/nexx.py
@@ -230,15 +230,18 @@ class NexxIE(InfoExtractor):

        azure_locator = stream_data['azureLocator']

-        AZURE_URL = 'http://nx%s%02d.akamaized.net/'
-
-        def get_cdn_shield_base(shield_type='', prefix='-p'):
+        def get_cdn_shield_base(shield_type='', static=False):
            for secure in ('', 's'):
                cdn_shield = stream_data.get('cdnShield%sHTTP%s' % (shield_type, secure.upper()))
                if cdn_shield:
                    return 'http%s://%s' % (secure, cdn_shield)
            else:
-                return AZURE_URL % (prefix, int(stream_data['azureAccount'].replace('nexxplayplus', '')))
+                if 'fb' in stream_data['azureAccount']:
+                    prefix = 'df' if static else 'f'
+                else:
+                    prefix = 'd' if static else 'p'
+                account = int(stream_data['azureAccount'].replace('nexxplayplus', '').replace('nexxplayfb', ''))
+                return 'http://nx-%s%02d.akamaized.net/' % (prefix, account)

        azure_stream_base = get_cdn_shield_base()
        is_ml = ',' in language
@@ -260,7 +263,7 @@ class NexxIE(InfoExtractor):
        formats.extend(self._extract_ism_formats(
            azure_manifest_url % '', video_id, ism_id='%s-mss' % cdn, fatal=False))

-        azure_progressive_base = get_cdn_shield_base('Prog', '-d')
+        azure_progressive_base = get_cdn_shield_base('Prog', True)
        azure_file_distribution = stream_data.get('azureFileDistribution')
        if azure_file_distribution:
            fds = azure_file_distribution.split(',')
--- a/youtube_dl/extractor/nick.py
+++ b/youtube_dl/extractor/nick.py
@@ -81,13 +81,26 @@ class NickIE(MTVServicesInfoExtractor):

 class NickBrIE(MTVServicesInfoExtractor):
    IE_NAME = 'nickelodeon:br'
-    _VALID_URL = r'https?://(?P<domain>(?:www\.)?nickjr|mundonick\.uol)\.com\.br/(?:programas/)?[^/]+/videos/(?:episodios/)?(?P<id>[^/?#.]+)'
+    _VALID_URL = r'''(?x)
+                    https?://
+                        (?:
+                            (?P<domain>(?:www\.)?nickjr|mundonick\.uol)\.com\.br|
+                            (?:www\.)?nickjr\.[a-z]{2}
+                        )
+                        /(?:programas/)?[^/]+/videos/(?:episodios/)?(?P<id>[^/?\#.]+)
+                    '''
    _TESTS = [{
        'url': 'http://www.nickjr.com.br/patrulha-canina/videos/210-labirinto-de-pipoca/',
        'only_matching': True,
    }, {
        'url': 'http://mundonick.uol.com.br/programas/the-loud-house/videos/muitas-irmas/7ljo9j',
        'only_matching': True,
+    }, {
+        'url': 'http://www.nickjr.nl/paw-patrol/videos/311-ge-wol-dig-om-terug-te-zijn/',
+        'only_matching': True,
+    }, {
+        'url': 'http://www.nickjr.de/blaze-und-die-monster-maschinen/videos/f6caaf8f-e4e8-4cc1-b489-9380d6dcd059/',
+        'only_matching': True,
    }]

    def _real_extract(self, url):
--- a/youtube_dl/extractor/nrk.py
+++ b/youtube_dl/extractor/nrk.py
@@ -237,7 +237,7 @@ class NRKTVIE(NRKBaseIE):
                            (?:/\d{2}-\d{2}-\d{4})?
                            (?:\#del=(?P<part_id>\d+))?
                    ''' % _EPISODE_RE
-    _API_HOST = 'psapi-ne.nrk.no'
+    _API_HOST = 'psapi-we.nrk.no'

    _TESTS = [{
        'url': 'https://tv.nrk.no/serie/20-spoersmaal-tv/MUHH48000314/23-05-2014',
--- a/youtube_dl/extractor/odnoklassniki.py
+++ b/youtube_dl/extractor/odnoklassniki.py
@@ -19,7 +19,18 @@ from ..utils import (


 class OdnoklassnikiIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:(?:www|m|mobile)\.)?(?:odnoklassniki|ok)\.ru/(?:video(?:embed)?|web-api/video/moviePlayer|live)/(?P<id>[\d-]+)'
+    _VALID_URL = r'''(?x)
+                https?://
+                    (?:(?:www|m|mobile)\.)?
+                    (?:odnoklassniki|ok)\.ru/
+                    (?:
+                        video(?:embed)?/|
+                        web-api/video/moviePlayer/|
+                        live/|
+                        dk\?.*?st\.mvId=
+                    )
+                    (?P<id>[\d-]+)
+                '''
    _TESTS = [{
        # metadata in JSON
        'url': 'http://ok.ru/video/20079905452',
@@ -101,6 +112,9 @@ class OdnoklassnikiIE(InfoExtractor):
    }, {
        'url': 'https://www.ok.ru/live/484531969818',
        'only_matching': True,
+    }, {
+        'url': 'https://m.ok.ru/dk?st.cmd=movieLayer&st.discId=863789452017&st.retLoc=friend&st.rtu=%2Fdk%3Fst.cmd%3DfriendMovies%26st.mode%3Down%26st.mrkId%3D%257B%2522uploadedMovieMarker%2522%253A%257B%2522marker%2522%253A%25221519410114503%2522%252C%2522hasMore%2522%253Atrue%257D%252C%2522sharedMovieMarker%2522%253A%257B%2522marker%2522%253Anull%252C%2522hasMore%2522%253Afalse%257D%257D%26st.friendId%3D561722190321%26st.frwd%3Don%26_prevCmd%3DfriendMovies%26tkn%3D7257&st.discType=MOVIE&st.mvId=863789452017&_prevCmd=friendMovies&tkn=3648#lst#',
+        'only_matching': True,
    }]

    def _real_extract(self, url):
--- a/youtube_dl/extractor/openload.py
+++ b/youtube_dl/extractor/openload.py
@@ -243,7 +243,7 @@ class PhantomJSwrapper(object):


 class OpenloadIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?(?:openload\.(?:co|io|link)|oload\.(?:tv|stream|site))/(?:f|embed)/(?P<id>[a-zA-Z0-9-_]+)'
+    _VALID_URL = r'https?://(?:www\.)?(?:openload\.(?:co|io|link)|oload\.(?:tv|stream|site|xyz))/(?:f|embed)/(?P<id>[a-zA-Z0-9-_]+)'

    _TESTS = [{
        'url': 'https://openload.co/f/kUEfGclsU9o',
@@ -298,6 +298,9 @@ class OpenloadIE(InfoExtractor):
    }, {
        'url': 'https://oload.stream/f/KnG-kKZdcfY',
        'only_matching': True,
+    }, {
+        'url': 'https://oload.xyz/f/WwRBpzW8Wtk',
+        'only_matching': True,
    }]

    _USER_AGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36'
@@ -334,10 +337,14 @@ class OpenloadIE(InfoExtractor):

        decoded_id = (get_element_by_id('streamurl', webpage) or
                      get_element_by_id('streamuri', webpage) or
-                      get_element_by_id('streamurj', webpage))
-
-        if not decoded_id:
-            raise ExtractorError('Can\'t find stream URL', video_id=video_id)
+                      get_element_by_id('streamurj', webpage) or
+                      self._search_regex(
+                          (r'>\s*([\w-]+~\d{10,}~\d+\.\d+\.0\.0~[\w-]+)\s*<',
+                           r'>\s*([\w~-]+~\d+\.\d+\.\d+\.\d+~[\w~-]+)',
+                           r'>\s*([\w-]+~\d{10,}~(?:[a-f\d]+:){2}:~[\w-]+)\s*<',
+                           r'>\s*([\w~-]+~[a-f0-9:]+~[\w~-]+)\s*<',
+                           r'>\s*([\w~-]+~[a-f0-9:]+~[\w~-]+)'), webpage,
+                          'stream URL'))

        video_url = 'https://openload.co/stream/%s?mime=true' % decoded_id

--- a/youtube_dl/extractor/pbs.py
+++ b/youtube_dl/extractor/pbs.py
@@ -505,7 +505,7 @@ class PBSIE(InfoExtractor):
            if player:
                video_info = self._parse_json(
                    self._search_regex(
-                        r'(?s)PBS\.videoData\s*=\s*({.+?});\n',
+                        [r'(?s)PBS\.videoData\s*=\s*({.+?});\n', r'window\.videoBridge\s*=\s*({.+?});'],
                        player, '%s video data' % page, default='{}'),
                    display_id, transform_source=js_to_json, fatal=False)
                if video_info:
@@ -513,10 +513,14 @@ class PBSIE(InfoExtractor):
                    if not info:
                        info = video_info
                if not chapters:
-                    for chapter_data in re.findall(r'(?s)chapters\.push\(({.*?})\)', player):
-                        chapter = self._parse_json(chapter_data, video_id, js_to_json, fatal=False)
-                        if not chapter:
-                            continue
+                    raw_chapters = video_info.get('chapters') or []
+                    if not raw_chapters:
+                        for chapter_data in re.findall(r'(?s)chapters\.push\(({.*?})\)', player):
+                            chapter = self._parse_json(chapter_data, video_id, js_to_json, fatal=False)
+                            if not chapter:
+                                continue
+                            raw_chapters.append(chapter)
+                    for chapter in raw_chapters:
                        start_time = float_or_none(chapter.get('start_time'), 1000)
                        duration = float_or_none(chapter.get('duration'), 1000)
                        if start_time is None or duration is None:
--- a/youtube_dl/extractor/picarto.py
+++ b/youtube_dl/extractor/picarto.py
@@ -0,0 +1,165 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+import time
+
+from .common import InfoExtractor
+from ..compat import compat_str
+from ..utils import (
+    ExtractorError,
+    js_to_json,
+    try_get,
+    update_url_query,
+    urlencode_postdata,
+)
+
+
+class PicartoIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www.)?picarto\.tv/(?P<id>[a-zA-Z0-9]+)'
+    _TEST = {
+        'url': 'https://picarto.tv/Setz',
+        'info_dict': {
+            'id': 'Setz',
+            'ext': 'mp4',
+            'title': 're:^Setz [0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}$',
+            'timestamp': int,
+            'is_live': True
+        },
+        'skip': 'Stream is offline',
+    }
+
+    @classmethod
+    def suitable(cls, url):
+        return False if PicartoVodIE.suitable(url) else super(PicartoIE, cls).suitable(url)
+
+    def _real_extract(self, url):
+        channel_id = self._match_id(url)
+        stream_page = self._download_webpage(url, channel_id)
+
+        if '>This channel does not exist' in stream_page:
+            raise ExtractorError(
+                'Channel %s does not exist' % channel_id, expected=True)
+
+        player = self._parse_json(
+            self._search_regex(
+                r'(?s)playerSettings\[\d+\]\s*=\s*(\{.+?\}\s*\n)', stream_page,
+                'player settings'),
+            channel_id, transform_source=js_to_json)
+
+        if player.get('online') is False:
+            raise ExtractorError('Stream is offline', expected=True)
+
+        cdn_data = self._download_json(
+            'https://picarto.tv/process/channel', channel_id,
+            data=urlencode_postdata({'loadbalancinginfo': channel_id}),
+            note='Downloading load balancing info')
+
+        def get_event(key):
+            return try_get(player, lambda x: x['event'][key], compat_str) or ''
+
+        params = {
+            'token': player.get('token') or '',
+            'ticket': get_event('ticket'),
+            'con': int(time.time() * 1000),
+            'type': get_event('ticket'),
+            'scope': get_event('scope'),
+        }
+
+        prefered_edge = cdn_data.get('preferedEdge')
+        default_tech = player.get('defaultTech')
+
+        formats = []
+
+        for edge in cdn_data['edges']:
+            edge_ep = edge.get('ep')
+            if not edge_ep or not isinstance(edge_ep, compat_str):
+                continue
+            edge_id = edge.get('id')
+            for tech in cdn_data['techs']:
+                tech_label = tech.get('label')
+                tech_type = tech.get('type')
+                preference = 0
+                if edge_id == prefered_edge:
+                    preference += 1
+                if tech_type == default_tech:
+                    preference += 1
+                format_id = []
+                if edge_id:
+                    format_id.append(edge_id)
+                if tech_type == 'application/x-mpegurl' or tech_label == 'HLS':
+                    format_id.append('hls')
+                    formats.extend(self._extract_m3u8_formats(
+                        update_url_query(
+                            'https://%s/hls/%s/index.m3u8'
+                            % (edge_ep, channel_id), params),
+                        channel_id, 'mp4', preference=preference,
+                        m3u8_id='-'.join(format_id), fatal=False))
+                    continue
+                elif tech_type == 'video/mp4' or tech_label == 'MP4':
+                    format_id.append('mp4')
+                    formats.append({
+                        'url': update_url_query(
+                            'https://%s/mp4/%s.mp4' % (edge_ep, channel_id),
+                            params),
+                        'format_id': '-'.join(format_id),
+                        'preference': preference,
+                    })
+                else:
+                    # rtmp format does not seem to work
+                    continue
+        self._sort_formats(formats)
+
+        mature = player.get('mature')
+        if mature is None:
+            age_limit = None
+        else:
+            age_limit = 18 if mature is True else 0
+
+        return {
+            'id': channel_id,
+            'title': self._live_title(channel_id),
+            'is_live': True,
+            'thumbnail': player.get('vodThumb'),
+            'age_limit': age_limit,
+            'formats': formats,
+        }
+
+
+class PicartoVodIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www.)?picarto\.tv/videopopout/(?P<id>[^/?#&]+)'
+    _TESTS = [{
+        'url': 'https://picarto.tv/videopopout/ArtofZod_2017.12.12.00.13.23.flv',
+        'md5': '3ab45ba4352c52ee841a28fb73f2d9ca',
+        'info_dict': {
+            'id': 'ArtofZod_2017.12.12.00.13.23.flv',
+            'ext': 'mp4',
+            'title': 'ArtofZod_2017.12.12.00.13.23.flv',
+            'thumbnail': r're:^https?://.*\.jpg'
+        },
+    }, {
+        'url': 'https://picarto.tv/videopopout/Plague',
+        'only_matching': True,
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+
+        webpage = self._download_webpage(url, video_id)
+
+        vod_info = self._parse_json(
+            self._search_regex(
+                r'(?s)#vod-player["\']\s*,\s*(\{.+?\})\s*\)', webpage,
+                video_id),
+            video_id, transform_source=js_to_json)
+
+        formats = self._extract_m3u8_formats(
+            vod_info['vod'], video_id, 'mp4', entry_protocol='m3u8_native',
+            m3u8_id='hls')
+        self._sort_formats(formats)
+
+        return {
+            'id': video_id,
+            'title': video_id,
+            'thumbnail': vod_info.get('vodThumb'),
+            'formats': formats,
+        }
--- a/youtube_dl/extractor/pluralsight.py
+++ b/youtube_dl/extractor/pluralsight.py
@@ -140,10 +140,10 @@ class PluralsightIE(PluralsightBaseIE):

            raise ExtractorError('Unable to log in')

-    def _get_subtitles(self, author, clip_id, lang, name, duration, video_id):
+    def _get_subtitles(self, author, clip_idx, lang, name, duration, video_id):
        captions_post = {
            'a': author,
-            'cn': clip_id,
+            'cn': clip_idx,
            'lc': lang,
            'm': name,
        }
@@ -195,13 +195,13 @@ class PluralsightIE(PluralsightBaseIE):

        author = qs.get('author', [None])[0]
        name = qs.get('name', [None])[0]
-        clip_id = qs.get('clip', [None])[0]
+        clip_idx = qs.get('clip', [None])[0]
        course_name = qs.get('course', [None])[0]

-        if any(not f for f in (author, name, clip_id, course_name,)):
+        if any(not f for f in (author, name, clip_idx, course_name,)):
            raise ExtractorError('Invalid URL', expected=True)

-        display_id = '%s-%s' % (name, clip_id)
+        display_id = '%s-%s' % (name, clip_idx)

        course = self._download_course(course_name, url, display_id)

@@ -217,7 +217,7 @@ class PluralsightIE(PluralsightBaseIE):
                        clip_index = clip_.get('index')
                    if clip_index is None:
                        continue
-                    if compat_str(clip_index) == clip_id:
+                    if compat_str(clip_index) == clip_idx:
                        clip = clip_
                        break

@@ -225,6 +225,7 @@ class PluralsightIE(PluralsightBaseIE):
            raise ExtractorError('Unable to resolve clip')

        title = clip['title']
+        clip_id = clip.get('clipName') or clip.get('name') or clip['clipId']

        QUALITIES = {
            'low': {'width': 640, 'height': 480},
@@ -277,7 +278,7 @@ class PluralsightIE(PluralsightBaseIE):
                clip_post = {
                    'author': author,
                    'includeCaptions': False,
-                    'clipIndex': int(clip_id),
+                    'clipIndex': int(clip_idx),
                    'courseName': course_name,
                    'locale': 'en',
                    'moduleName': name,
@@ -330,10 +331,10 @@ class PluralsightIE(PluralsightBaseIE):

        # TODO: other languages?
        subtitles = self.extract_subtitles(
-            author, clip_id, 'en', name, duration, display_id)
+            author, clip_idx, 'en', name, duration, display_id)

        return {
-            'id': clip.get('clipName') or clip['name'],
+            'id': clip_id,
            'title': title,
            'duration': duration,
            'creator': author,
--- a/youtube_dl/extractor/pornflip.py
+++ b/youtube_dl/extractor/pornflip.py
@@ -14,7 +14,7 @@ from ..utils import (


 class PornFlipIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?pornflip\.com/(?:v|embed)/(?P<id>[0-9A-Za-z-]{11})'
+    _VALID_URL = r'https?://(?:www\.)?pornflip\.com/(?:v|embed)/(?P<id>[^/?#&]+)'
    _TESTS = [{
        'url': 'https://www.pornflip.com/v/wz7DfNhMmep',
        'md5': '98c46639849145ae1fd77af532a9278c',
@@ -40,6 +40,9 @@ class PornFlipIE(InfoExtractor):
    }, {
        'url': 'https://www.pornflip.com/embed/EkRD6-vS2-s',
        'only_matching': True,
+    }, {
+        'url': 'https://www.pornflip.com/v/NG9q6Pb_iK8',
+        'only_matching': True,
    }]

    def _real_extract(self, url):
--- a/youtube_dl/extractor/pornhub.py
+++ b/youtube_dl/extractor/pornhub.py
@@ -33,7 +33,7 @@ class PornHubIE(InfoExtractor):
    _VALID_URL = r'''(?x)
                    https?://
                        (?:
-                            (?:[a-z]+\.)?pornhub\.com/(?:(?:view_video\.php|video/show)\?viewkey=|embed/)|
+                            (?:[^/]+\.)?pornhub\.com/(?:(?:view_video\.php|video/show)\?viewkey=|embed/)|
                            (?:www\.)?thumbzilla\.com/video/
                        )
                        (?P<id>[\da-z]+)
@@ -264,7 +264,7 @@ class PornHubPlaylistBaseIE(InfoExtractor):


 class PornHubPlaylistIE(PornHubPlaylistBaseIE):
-    _VALID_URL = r'https?://(?:www\.)?pornhub\.com/playlist/(?P<id>\d+)'
+    _VALID_URL = r'https?://(?:[^/]+\.)?pornhub\.com/playlist/(?P<id>\d+)'
    _TESTS = [{
        'url': 'http://www.pornhub.com/playlist/4667351',
        'info_dict': {
@@ -272,11 +272,14 @@ class PornHubPlaylistIE(PornHubPlaylistBaseIE):
            'title': 'Nataly Hot',
        },
        'playlist_mincount': 2,
+    }, {
+        'url': 'https://de.pornhub.com/playlist/4667351',
+        'only_matching': True,
    }]


 class PornHubUserVideosIE(PornHubPlaylistBaseIE):
-    _VALID_URL = r'https?://(?:www\.)?pornhub\.com/(?:user|channel)s/(?P<id>[^/]+)/videos'
+    _VALID_URL = r'https?://(?:[^/]+\.)?pornhub\.com/(?:user|channel)s/(?P<id>[^/]+)/videos'
    _TESTS = [{
        'url': 'http://www.pornhub.com/users/zoe_ph/videos/public',
        'info_dict': {
@@ -305,6 +308,9 @@ class PornHubUserVideosIE(PornHubPlaylistBaseIE):
        # Most Viewed Videos
        'url': 'https://www.pornhub.com/channels/povd/videos?o=vi',
        'only_matching': True,
+    }, {
+        'url': 'http://www.pornhub.com/users/zoe_ph/videos/public',
+        'only_matching': True,
    }]

    def _real_extract(self, url):
--- a/youtube_dl/extractor/reddit.py
+++ b/youtube_dl/extractor/reddit.py
@@ -47,7 +47,7 @@ class RedditIE(InfoExtractor):


 class RedditRIE(InfoExtractor):
-    _VALID_URL = r'(?P<url>https?://(?:www\.)?reddit\.com/r/[^/]+/comments/(?P<id>[^/?#&]+))'
+    _VALID_URL = r'(?P<url>https?://(?:[^/]+\.)?reddit\.com/r/[^/]+/comments/(?P<id>[^/?#&]+))'
    _TESTS = [{
        'url': 'https://www.reddit.com/r/videos/comments/6rrwyj/that_small_heart_attack/',
        'info_dict': {
@@ -74,6 +74,10 @@ class RedditRIE(InfoExtractor):
        # imgur
        'url': 'https://www.reddit.com/r/MadeMeSmile/comments/6t7wi5/wait_for_it/',
        'only_matching': True,
+    }, {
+        # imgur @ old reddit
+        'url': 'https://old.reddit.com/r/MadeMeSmile/comments/6t7wi5/wait_for_it/',
+        'only_matching': True,
    }, {
        # streamable
        'url': 'https://www.reddit.com/r/videos/comments/6t7sg9/comedians_hilarious_joke_about_the_guam_flag/',
@@ -82,6 +86,10 @@ class RedditRIE(InfoExtractor):
        # youtube
        'url': 'https://www.reddit.com/r/videos/comments/6t75wq/southern_man_tries_to_speak_without_an_accent/',
        'only_matching': True,
+    }, {
+        # reddit video @ nm reddit
+        'url': 'https://nm.reddit.com/r/Cricket/comments/8idvby/lousy_cameraman_finds_himself_in_cairns_line_of/',
+        'only_matching': True,
    }]

    def _real_extract(self, url):
--- a/youtube_dl/extractor/rentv.py
+++ b/youtube_dl/extractor/rentv.py
@@ -3,6 +3,10 @@ from __future__ import unicode_literals

 from .common import InfoExtractor
 from ..compat import compat_str
+from ..utils import (
+    determine_ext,
+    int_or_none,
+)


 class RENTVIE(InfoExtractor):
@@ -13,7 +17,9 @@ class RENTVIE(InfoExtractor):
        'info_dict': {
            'id': '118577',
            'ext': 'mp4',
-            'title': 'Документальный спецпроект: "Промывка мозгов. Технологии XXI века"'
+            'title': 'Документальный спецпроект: "Промывка мозгов. Технологии XXI века"',
+            'timestamp': 1472230800,
+            'upload_date': '20160826',
        }
    }, {
        'url': 'http://ren.tv/player/118577',
@@ -26,9 +32,33 @@ class RENTVIE(InfoExtractor):
    def _real_extract(self, url):
        video_id = self._match_id(url)
        webpage = self._download_webpage('http://ren.tv/player/' + video_id, video_id)
-        jw_config = self._parse_json(self._search_regex(
-            r'config\s*=\s*({.+});', webpage, 'jw config'), video_id)
-        return self._parse_jwplayer_data(jw_config, video_id, m3u8_id='hls')
+        config = self._parse_json(self._search_regex(
+            r'config\s*=\s*({.+})\s*;', webpage, 'config'), video_id)
+        title = config['title']
+        formats = []
+        for video in config['src']:
+            src = video.get('src')
+            if not src or not isinstance(src, compat_str):
+                continue
+            ext = determine_ext(src)
+            if ext == 'm3u8':
+                formats.extend(self._extract_m3u8_formats(
+                    src, video_id, 'mp4', entry_protocol='m3u8_native',
+                    m3u8_id='hls', fatal=False))
+            else:
+                formats.append({
+                    'url': src,
+                })
+        self._sort_formats(formats)
+        return {
+            'id': video_id,
+            'title': title,
+            'description': config.get('description'),
+            'thumbnail': config.get('image'),
+            'duration': int_or_none(config.get('duration')),
+            'timestamp': int_or_none(config.get('date')),
+            'formats': formats,
+        }


 class RENTVArticleIE(InfoExtractor):
--- a/youtube_dl/extractor/smotri.py
+++ b/youtube_dl/extractor/smotri.py
@@ -310,6 +310,7 @@ class SmotriBroadcastIE(InfoExtractor):
    IE_DESC = 'Smotri.com broadcasts'
    IE_NAME = 'smotri:broadcast'
    _VALID_URL = r'https?://(?:www\.)?(?P<url>smotri\.com/live/(?P<id>[^/]+))/?.*'
+    _NETRC_MACHINE = 'smotri'

    def _real_extract(self, url):
        mobj = re.match(self._VALID_URL, url)
@@ -352,17 +353,18 @@ class SmotriBroadcastIE(InfoExtractor):
            adult_content = False

        ticket = self._html_search_regex(
-            r"window\.broadcast_control\.addFlashVar\('file'\s*,\s*'([^']+)'\)",
-            broadcast_page, 'broadcast ticket')
+            (r'data-user-file=(["\'])(?P<ticket>(?!\1).+)\1',
+             r"window\.broadcast_control\.addFlashVar\('file'\s*,\s*'(?P<ticket>[^']+)'\)"),
+            broadcast_page, 'broadcast ticket', group='ticket')

-        url = 'http://smotri.com/broadcast/view/url/?ticket=%s' % ticket
+        broadcast_url = 'http://smotri.com/broadcast/view/url/?ticket=%s' % ticket

        broadcast_password = self._downloader.params.get('videopassword')
        if broadcast_password:
-            url += '&pass=%s' % hashlib.md5(broadcast_password.encode('utf-8')).hexdigest()
+            broadcast_url += '&pass=%s' % hashlib.md5(broadcast_password.encode('utf-8')).hexdigest()

        broadcast_json_page = self._download_webpage(
-            url, broadcast_id, 'Downloading broadcast JSON')
+            broadcast_url, broadcast_id, 'Downloading broadcast JSON')

        try:
            broadcast_json = json.loads(broadcast_json_page)
--- a/youtube_dl/extractor/spike.py
+++ b/youtube_dl/extractor/spike.py
@@ -1,55 +1,46 @@
 from __future__ import unicode_literals

-import re
-
 from .mtv import MTVServicesInfoExtractor


-class SpikeIE(MTVServicesInfoExtractor):
-    _VALID_URL = r'https?://(?:[^/]+\.)?spike\.com/[^/]+/[\da-z]{6}(?:[/?#&]|$)'
+class BellatorIE(MTVServicesInfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?bellator\.com/[^/]+/[\da-z]{6}(?:[/?#&]|$)'
    _TESTS = [{
-        'url': 'http://www.spike.com/video-clips/lhtu8m/auction-hunters-can-allen-ride-a-hundred-year-old-motorcycle',
-        'md5': '1a9265f32b0c375793d6c4ce45255256',
+        'url': 'http://www.bellator.com/fight/atwr7k/bellator-158-michael-page-vs-evangelista-cyborg',
        'info_dict': {
-            'id': 'b9c8221a-4e50-479a-b86d-3333323e38ba',
+            'id': 'b55e434e-fde1-4a98-b7cc-92003a034de4',
            'ext': 'mp4',
-            'title': 'Auction Hunters|December 27, 2013|4|414|Can Allen Ride A Hundred Year-Old Motorcycle?',
-            'description': 'md5:fbed7e82ed5fad493615b3094a9499cb',
-            'timestamp': 1388120400,
-            'upload_date': '20131227',
+            'title': 'Douglas Lima vs. Paul Daley - Round 1',
+            'description': 'md5:805a8dd29310fd611d32baba2f767885',
+        },
+        'params': {
+            # m3u8 download
+            'skip_download': True,
        },
    }, {
-        'url': 'http://www.spike.com/full-episodes/j830qm/lip-sync-battle-joel-mchale-vs-jim-rash-season-2-ep-209',
-        'md5': 'b25c6f16418aefb9ad5a6cae2559321f',
+        'url': 'http://www.bellator.com/video-clips/bw6k7n/bellator-158-foundations-michael-venom-page',
+        'only_matching': True,
+    }]
+
+    _FEED_URL = 'http://www.spike.com/feeds/mrss/'
+    _GEO_COUNTRIES = ['US']
+
+
+class ParamountNetworkIE(MTVServicesInfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?paramountnetwork\.com/[^/]+/[\da-z]{6}(?:[/?#&]|$)'
+    _TESTS = [{
+        'url': 'http://www.paramountnetwork.com/episodes/j830qm/lip-sync-battle-joel-mchale-vs-jim-rash-season-2-ep-13',
        'info_dict': {
            'id': '37ace3a8-1df6-48be-85b8-38df8229e241',
            'ext': 'mp4',
            'title': 'Lip Sync Battle|April 28, 2016|2|209|Joel McHale Vs. Jim Rash|Act 1',
            'description': 'md5:a739ca8f978a7802f67f8016d27ce114',
        },
-    }, {
-        'url': 'http://www.spike.com/video-clips/lhtu8m/',
-        'only_matching': True,
-    }, {
-        'url': 'http://www.spike.com/video-clips/lhtu8m',
-        'only_matching': True,
-    }, {
-        'url': 'http://bellator.spike.com/fight/atwr7k/bellator-158-michael-page-vs-evangelista-cyborg',
-        'only_matching': True,
-    }, {
-        'url': 'http://bellator.spike.com/video-clips/bw6k7n/bellator-158-foundations-michael-venom-page',
-        'only_matching': True,
+        'params': {
+            # m3u8 download
+            'skip_download': True,
+        },
    }]

-    _FEED_URL = 'http://www.spike.com/feeds/mrss/'
-    _MOBILE_TEMPLATE = 'http://m.spike.com/videos/video.rbml?id=%s'
-    _CUSTOM_URL_REGEX = re.compile(r'spikenetworkapp://([^/]+/[-a-fA-F0-9]+)')
+    _FEED_URL = 'http://www.paramountnetwork.com/feeds/mrss/'
    _GEO_COUNTRIES = ['US']
-
-    def _extract_mgid(self, webpage):
-        mgid = super(SpikeIE, self)._extract_mgid(webpage)
-        if mgid is None:
-            url_parts = self._search_regex(self._CUSTOM_URL_REGEX, webpage, 'episode_id')
-            video_type, episode_id = url_parts.split('/', 1)
-            mgid = 'mgid:arc:{0}:spike.com:{1}'.format(video_type, episode_id)
-        return mgid
--- a/youtube_dl/extractor/steam.py
+++ b/youtube_dl/extractor/steam.py
@@ -75,6 +75,9 @@ class SteamIE(InfoExtractor):
            gameID = m.group('gameID')
            playlist_id = gameID
            videourl = self._VIDEO_PAGE_TEMPLATE % playlist_id
+
+        self._set_cookie('steampowered.com', 'mature_content', '1')
+
        webpage = self._download_webpage(videourl, playlist_id)

        if re.search('<h2>Please enter your birth date to continue:</h2>', webpage) is not None:
--- a/youtube_dl/extractor/svt.py
+++ b/youtube_dl/extractor/svt.py
@@ -4,11 +4,17 @@ from __future__ import unicode_literals
 import re

 from .common import InfoExtractor
+from ..compat import (
+    compat_parse_qs,
+    compat_urllib_parse_urlparse,
+)
 from ..utils import (
    determine_ext,
    dict_get,
    int_or_none,
    try_get,
+    urljoin,
+    compat_str,
 )


@@ -16,6 +22,8 @@ class SVTBaseIE(InfoExtractor):
    _GEO_COUNTRIES = ['SE']

    def _extract_video(self, video_info, video_id):
+        is_live = dict_get(video_info, ('live', 'simulcast'), default=False)
+        m3u8_protocol = 'm3u8' if is_live else 'm3u8_native'
        formats = []
        for vr in video_info['videoReferences']:
            player_type = vr.get('playerType') or vr.get('format')
@@ -24,7 +32,7 @@ class SVTBaseIE(InfoExtractor):
            if ext == 'm3u8':
                formats.extend(self._extract_m3u8_formats(
                    vurl, video_id,
-                    ext='mp4', entry_protocol='m3u8_native',
+                    ext='mp4', entry_protocol=m3u8_protocol,
                    m3u8_id=player_type, fatal=False))
            elif ext == 'f4m':
                formats.extend(self._extract_f4m_formats(
@@ -84,6 +92,7 @@ class SVTBaseIE(InfoExtractor):
            'season_number': season_number,
            'episode': episode,
            'episode_number': episode_number,
+            'is_live': is_live,
        }


@@ -122,9 +131,13 @@ class SVTIE(SVTBaseIE):
        return info_dict


-class SVTPlayIE(SVTBaseIE):
+class SVTPlayBaseIE(SVTBaseIE):
+    _SVTPLAY_RE = r'root\s*\[\s*(["\'])_*svtplay\1\s*\]\s*=\s*(?P<json>{.+?})\s*;\s*\n'
+
+
+class SVTPlayIE(SVTPlayBaseIE):
    IE_DESC = 'SVT Play and Öppet arkiv'
-    _VALID_URL = r'https?://(?:www\.)?(?:svtplay|oppetarkiv)\.se/(?:video|klipp)/(?P<id>[0-9]+)'
+    _VALID_URL = r'https?://(?:www\.)?(?:svtplay|oppetarkiv)\.se/(?:video|klipp|kanaler)/(?P<id>[^/?#&]+)'
    _TESTS = [{
        'url': 'http://www.svtplay.se/video/5996901/flygplan-till-haile-selassie/flygplan-till-haile-selassie-2',
        'md5': '2b6704fe4a28801e1a098bbf3c5ac611',
@@ -148,6 +161,9 @@ class SVTPlayIE(SVTBaseIE):
    }, {
        'url': 'http://www.svtplay.se/klipp/9023742/stopptid-om-bjorn-borg',
        'only_matching': True,
+    }, {
+        'url': 'https://www.svtplay.se/kanaler/svt1',
+        'only_matching': True,
    }]

    def _real_extract(self, url):
@@ -157,12 +173,16 @@ class SVTPlayIE(SVTBaseIE):

        data = self._parse_json(
            self._search_regex(
-                r'root\["__svtplay"\]\s*=\s*([^;]+);',
-                webpage, 'embedded data', default='{}'),
+                self._SVTPLAY_RE, webpage, 'embedded data', default='{}',
+                group='json'),
            video_id, fatal=False)

        thumbnail = self._og_search_thumbnail(webpage)

+        def adjust_title(info):
+            if info['is_live']:
+                info['title'] = self._live_title(info['title'])
+
        if data:
            video_info = try_get(
                data, lambda x: x['context']['dispatcher']['stores']['VideoTitlePageStore']['data']['video'],
@@ -173,6 +193,7 @@ class SVTPlayIE(SVTBaseIE):
                    'title': data['context']['dispatcher']['stores']['MetaStore']['title'],
                    'thumbnail': thumbnail,
                })
+                adjust_title(info_dict)
                return info_dict

        video_id = self._search_regex(
@@ -188,4 +209,86 @@ class SVTPlayIE(SVTBaseIE):
                info_dict['title'] = re.sub(
                    r'\s*\|\s*.+?$', '',
                    info_dict.get('episode') or self._og_search_title(webpage))
+            adjust_title(info_dict)
            return info_dict
+
+
+class SVTSeriesIE(SVTPlayBaseIE):
+    _VALID_URL = r'https?://(?:www\.)?svtplay\.se/(?P<id>[^/?&#]+)'
+    _TESTS = [{
+        'url': 'https://www.svtplay.se/rederiet',
+        'info_dict': {
+            'id': 'rederiet',
+            'title': 'Rederiet',
+            'description': 'md5:505d491a58f4fcf6eb418ecab947e69e',
+        },
+        'playlist_mincount': 318,
+    }, {
+        'url': 'https://www.svtplay.se/rederiet?tab=sasong2',
+        'info_dict': {
+            'id': 'rederiet-sasong2',
+            'title': 'Rederiet - Säsong 2',
+            'description': 'md5:505d491a58f4fcf6eb418ecab947e69e',
+        },
+        'playlist_count': 12,
+    }]
+
+    @classmethod
+    def suitable(cls, url):
+        return False if SVTIE.suitable(url) or SVTPlayIE.suitable(url) else super(SVTSeriesIE, cls).suitable(url)
+
+    def _real_extract(self, url):
+        series_id = self._match_id(url)
+
+        qs = compat_parse_qs(compat_urllib_parse_urlparse(url).query)
+        season_slug = qs.get('tab', [None])[0]
+
+        if season_slug:
+            series_id += '-%s' % season_slug
+
+        webpage = self._download_webpage(
+            url, series_id, 'Downloading series page')
+
+        root = self._parse_json(
+            self._search_regex(
+                self._SVTPLAY_RE, webpage, 'content', group='json'),
+            series_id)
+
+        season_name = None
+
+        entries = []
+        for season in root['relatedVideoContent']['relatedVideosAccordion']:
+            if not isinstance(season, dict):
+                continue
+            if season_slug:
+                if season.get('slug') != season_slug:
+                    continue
+                season_name = season.get('name')
+            videos = season.get('videos')
+            if not isinstance(videos, list):
+                continue
+            for video in videos:
+                content_url = video.get('contentUrl')
+                if not content_url or not isinstance(content_url, compat_str):
+                    continue
+                entries.append(
+                    self.url_result(
+                        urljoin(url, content_url),
+                        ie=SVTPlayIE.ie_key(),
+                        video_title=video.get('title')
+                    ))
+
+        metadata = root.get('metaData')
+        if not isinstance(metadata, dict):
+            metadata = {}
+
+        title = metadata.get('title')
+        season_name = season_name or season_slug
+
+        if title and season_name:
+            title = '%s - %s' % (title, season_name)
+        elif season_slug:
+            title = season_slug
+
+        return self.playlist_result(
+            entries, series_id, title, metadata.get('description'))
--- a/youtube_dl/extractor/teamcoco.py
+++ b/youtube_dl/extractor/teamcoco.py
@@ -1,35 +1,34 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import binascii
-import re
 import json

 from .common import InfoExtractor
-from ..compat import (
-    compat_b64decode,
-    compat_ord,
-)
 from ..utils import (
-    ExtractorError,
-    qualities,
    determine_ext,
+    ExtractorError,
+    int_or_none,
+    mimetype2ext,
+    parse_duration,
+    parse_iso8601,
+    qualities,
 )


 class TeamcocoIE(InfoExtractor):
-    _VALID_URL = r'https?://teamcoco\.com/video/(?P<video_id>[0-9]+)?/?(?P<display_id>.*)'
+    _VALID_URL = r'https?://teamcoco\.com/video/(?P<id>[^/?#]+)'
    _TESTS = [
        {
-            'url': 'http://teamcoco.com/video/80187/conan-becomes-a-mary-kay-beauty-consultant',
-            'md5': '3f7746aa0dc86de18df7539903d399ea',
+            'url': 'http://teamcoco.com/video/mary-kay-remote',
+            'md5': '55d532f81992f5c92046ad02fec34d7d',
            'info_dict': {
                'id': '80187',
                'ext': 'mp4',
                'title': 'Conan Becomes A Mary Kay Beauty Consultant',
                'description': 'Mary Kay is perhaps the most trusted name in female beauty, so of course Conan is a natural choice to sell their products.',
-                'duration': 504,
-                'age_limit': 0,
+                'duration': 495.0,
+                'upload_date': '20140402',
+                'timestamp': 1396407600,
            }
        }, {
            'url': 'http://teamcoco.com/video/louis-ck-interview-george-w-bush',
@@ -40,7 +39,8 @@ class TeamcocoIE(InfoExtractor):
                'description': 'Louis C.K. got starstruck by George W. Bush, so what? Part one.',
                'title': 'Louis C.K. Interview Pt. 1 11/3/11',
                'duration': 288,
-                'age_limit': 0,
+                'upload_date': '20111104',
+                'timestamp': 1320405840,
            }
        }, {
            'url': 'http://teamcoco.com/video/timothy-olyphant-drinking-whiskey',
@@ -49,6 +49,8 @@ class TeamcocoIE(InfoExtractor):
                'ext': 'mp4',
                'title': 'Timothy Olyphant Raises A Toast To “Justified”',
                'description': 'md5:15501f23f020e793aeca761205e42c24',
+                'upload_date': '20150415',
+                'timestamp': 1429088400,
            },
            'params': {
                'skip_download': True,  # m3u8 downloads
@@ -63,110 +65,93 @@ class TeamcocoIE(InfoExtractor):
            },
            'params': {
                'skip_download': True,  # m3u8 downloads
-            }
+            },
+            'skip': 'This video is no longer available.',
        }
    ]
-    _VIDEO_ID_REGEXES = (
-        r'"eVar42"\s*:\s*(\d+)',
-        r'Ginger\.TeamCoco\.openInApp\("video",\s*"([^"]+)"',
-        r'"id_not"\s*:\s*(\d+)'
-    )
+
+    def _graphql_call(self, query_template, object_type, object_id):
+        find_object = 'find' + object_type
+        return self._download_json(
+            'http://teamcoco.com/graphql/', object_id, data=json.dumps({
+                'query': query_template % (find_object, object_id)
+            }))['data'][find_object]

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
+        display_id = self._match_id(url)

-        display_id = mobj.group('display_id')
-        webpage, urlh = self._download_webpage_handle(url, display_id)
-        if 'src=expired' in urlh.geturl():
-            raise ExtractorError('This video is expired.', expected=True)
+        response = self._graphql_call('''{
+  %s(slug: "video/%s") {
+    ... on RecordSlug {
+      record {
+        id
+        title
+        teaser
+        publishOn
+        thumb {
+          preview
+        }
+        tags {
+          name
+        }
+        duration
+      }
+    }
+    ... on NotFoundSlug {
+      status
+    }
+  }
+}''', 'Slug', display_id)
+        if response.get('status'):
+            raise ExtractorError('This video is no longer available.', expected=True)

-        video_id = mobj.group('video_id')
-        if not video_id:
-            video_id = self._html_search_regex(
-                self._VIDEO_ID_REGEXES, webpage, 'video id')
+        record = response['record']
+        video_id = record['id']

-        data = None
-
-        preload_codes = self._html_search_regex(
-            r'(function.+)setTimeout\(function\(\)\{playlist',
-            webpage, 'preload codes')
-        base64_fragments = re.findall(r'"([a-zA-Z0-9+/=]+)"', preload_codes)
-        base64_fragments.remove('init')
-
-        def _check_sequence(cur_fragments):
-            if not cur_fragments:
-                return
-            for i in range(len(cur_fragments)):
-                cur_sequence = (''.join(cur_fragments[i:] + cur_fragments[:i])).encode('ascii')
-                try:
-                    raw_data = compat_b64decode(cur_sequence)
-                    if compat_ord(raw_data[0]) == compat_ord('{'):
-                        return json.loads(raw_data.decode('utf-8'))
-                except (TypeError, binascii.Error, UnicodeDecodeError, ValueError):
-                    continue
-
-        def _check_data():
-            for i in range(len(base64_fragments) + 1):
-                for j in range(i, len(base64_fragments) + 1):
-                    data = _check_sequence(base64_fragments[:i] + base64_fragments[j:])
-                    if data:
-                        return data
-
-        self.to_screen('Try to compute possible data sequence. This may take some time.')
-        data = _check_data()
-
-        if not data:
-            raise ExtractorError(
-                'Preload information could not be extracted', expected=True)
+        srcs = self._graphql_call('''{
+  %s(id: "%s") {
+    src
+  }
+}''', 'RecordVideoSource', video_id)['src']

        formats = []
-        get_quality = qualities(['500k', '480p', '1000k', '720p', '1080p'])
-        for filed in data['files']:
-            if determine_ext(filed['url']) == 'm3u8':
-                # compat_urllib_parse.urljoin does not work here
-                if filed['url'].startswith('/'):
-                    m3u8_url = 'http://ht.cdn.turner.com/tbs/big/teamcoco' + filed['url']
-                else:
-                    m3u8_url = filed['url']
-                m3u8_formats = self._extract_m3u8_formats(
-                    m3u8_url, video_id, ext='mp4')
-                for m3u8_format in m3u8_formats:
-                    if m3u8_format not in formats:
-                        formats.append(m3u8_format)
-            elif determine_ext(filed['url']) == 'f4m':
-                # TODO Correct f4m extraction
+        get_quality = qualities(['low', 'sd', 'hd', 'uhd'])
+        for format_id, src in srcs.items():
+            if not isinstance(src, dict):
                continue
+            src_url = src.get('src')
+            if not src_url:
+                continue
+            ext = determine_ext(src_url, mimetype2ext(src.get('type')))
+            if format_id == 'hls' or ext == 'm3u8':
+                # compat_urllib_parse.urljoin does not work here
+                if src_url.startswith('/'):
+                    src_url = 'http://ht.cdn.turner.com/tbs/big/teamcoco' + src_url
+                formats.extend(self._extract_m3u8_formats(
+                    src_url, video_id, 'mp4', m3u8_id=format_id, fatal=False))
            else:
-                if filed['url'].startswith('/mp4:protected/'):
+                if src_url.startswith('/mp4:protected/'):
                    # TODO Correct extraction for these files
                    continue
-                m_format = re.search(r'(\d+(k|p))\.mp4', filed['url'])
-                if m_format is not None:
-                    format_id = m_format.group(1)
-                else:
-                    format_id = filed['bitrate']
-                tbr = (
-                    int(filed['bitrate'])
-                    if filed['bitrate'].isdigit()
-                    else None)
+                tbr = int_or_none(self._search_regex(
+                    r'(\d+)k\.mp4', src_url, 'tbr', default=None))

                formats.append({
-                    'url': filed['url'],
-                    'ext': 'mp4',
+                    'url': src_url,
+                    'ext': ext,
                    'tbr': tbr,
                    'format_id': format_id,
                    'quality': get_quality(format_id),
                })
-
        self._sort_formats(formats)

        return {
            'id': video_id,
            'display_id': display_id,
            'formats': formats,
-            'title': data['title'],
-            'thumbnail': data.get('thumb', {}).get('href'),
-            'description': data.get('teaser'),
-            'duration': data.get('duration'),
-            'age_limit': self._family_friendly_search(webpage),
+            'title': record['title'],
+            'thumbnail': record.get('thumb', {}).get('preview'),
+            'description': record.get('teaser'),
+            'duration': parse_duration(record.get('duration')),
+            'timestamp': parse_iso8601(record.get('publishOn')),
        }
--- a/youtube_dl/extractor/tube8.py
+++ b/youtube_dl/extractor/tube8.py
@@ -31,6 +31,12 @@ class Tube8IE(KeezMoviesIE):
        'only_matching': True,
    }]

+    @staticmethod
+    def _extract_urls(webpage):
+        return re.findall(
+            r'<iframe[^>]+\bsrc=["\']((?:https?:)?//(?:www\.)?tube8\.com/embed/(?:[^/]+/)+\d+)',
+            webpage)
+
    def _real_extract(self, url):
        webpage, info = self._extract_info(url)

--- a/youtube_dl/extractor/tunein.py
+++ b/youtube_dl/extractor/tunein.py
@@ -62,7 +62,7 @@ class TuneInBaseIE(InfoExtractor):

        return {
            'id': content_id,
-            'title': title,
+            'title': self._live_title(title) if is_live else title,
            'formats': formats,
            'thumbnail': thumbnail,
            'location': location,
--- a/youtube_dl/extractor/tvplay.py
+++ b/youtube_dl/extractor/tvplay.py
@@ -227,14 +227,16 @@ class TVPlayIE(InfoExtractor):

    def _real_extract(self, url):
        url, smuggled_data = unsmuggle_url(url, {})
-        self._initialize_geo_bypass(smuggled_data.get('geo_countries'))
+        self._initialize_geo_bypass({
+            'countries': smuggled_data.get('geo_countries'),
+        })

        video_id = self._match_id(url)
        geo_country = self._search_regex(
            r'https?://[^/]+\.([a-z]{2})', url,
            'geo country', default=None)
        if geo_country:
-            self._initialize_geo_bypass([geo_country.upper()])
+            self._initialize_geo_bypass({'countries': [geo_country.upper()]})
        video = self._download_json(
            'http://playapi.mtgx.tv/v3/videos/%s' % video_id, video_id, 'Downloading video JSON')

--- a/youtube_dl/extractor/twitch.py
+++ b/youtube_dl/extractor/twitch.py
@@ -8,6 +8,7 @@ import random
 from .common import InfoExtractor
 from ..compat import (
    compat_HTTPError,
+    compat_kwargs,
    compat_parse_qs,
    compat_str,
    compat_urllib_parse_urlencode,
@@ -16,11 +17,14 @@ from ..compat import (
 from ..utils import (
    clean_html,
    ExtractorError,
+    float_or_none,
    int_or_none,
-    js_to_json,
    orderedSet,
    parse_duration,
    parse_iso8601,
+    qualities,
+    try_get,
+    unified_timestamp,
    update_url_query,
    urlencode_postdata,
    urljoin,
@@ -28,7 +32,7 @@ from ..utils import (


 class TwitchBaseIE(InfoExtractor):
-    _VALID_URL_BASE = r'https?://(?:(?:www|go)\.)?twitch\.tv'
+    _VALID_URL_BASE = r'https?://(?:(?:www|go|m)\.)?twitch\.tv'

    _API_BASE = 'https://api.twitch.tv'
    _USHER_BASE = 'https://usher.ttvnw.net'
@@ -45,10 +49,11 @@ class TwitchBaseIE(InfoExtractor):
                '%s returned error: %s - %s' % (self.IE_NAME, error, response.get('message')),
                expected=True)

-    def _call_api(self, path, item_id, note):
+    def _call_api(self, path, item_id, *args, **kwargs):
+        kwargs.setdefault('headers', {})['Client-ID'] = self._CLIENT_ID
        response = self._download_json(
-            '%s/%s' % (self._API_BASE, path), item_id, note,
-            headers={'Client-ID': self._CLIENT_ID})
+            '%s/%s' % (self._API_BASE, path), item_id,
+            *args, **compat_kwargs(kwargs))
        self._handle_error(response)
        return response

@@ -168,6 +173,13 @@ class TwitchItemBaseIE(TwitchBaseIE):
        return self.playlist_result(entries, info['id'], info['title'])

    def _extract_info(self, info):
+        status = info.get('status')
+        if status == 'recording':
+            is_live = True
+        elif status == 'recorded':
+            is_live = False
+        else:
+            is_live = None
        return {
            'id': info['_id'],
            'title': info.get('title') or 'Untitled Broadcast',
@@ -178,6 +190,7 @@ class TwitchItemBaseIE(TwitchBaseIE):
            'uploader_id': info.get('channel', {}).get('name'),
            'timestamp': parse_iso8601(info.get('recorded_at')),
            'view_count': int_or_none(info.get('views')),
+            'is_live': is_live,
        }

    def _real_extract(self, url):
@@ -226,7 +239,7 @@ class TwitchVodIE(TwitchItemBaseIE):
    _VALID_URL = r'''(?x)
                    https?://
                        (?:
-                            (?:(?:www|go)\.)?twitch\.tv/(?:[^/]+/v|videos)/|
+                            (?:(?:www|go|m)\.)?twitch\.tv/(?:[^/]+/v|videos)/|
                            player\.twitch\.tv/\?.*?\bvideo=v
                        )
                        (?P<id>\d+)
@@ -279,6 +292,9 @@ class TwitchVodIE(TwitchItemBaseIE):
    }, {
        'url': 'https://www.twitch.tv/videos/6528877',
        'only_matching': True,
+    }, {
+        'url': 'https://m.twitch.tv/beagsandjam/v/247478721',
+        'only_matching': True,
    }]

    def _real_extract(self, url):
@@ -390,14 +406,17 @@ class TwitchProfileIE(TwitchPlaylistBaseIE):
    _VALID_URL = r'%s/(?P<id>[^/]+)/profile/?(?:\#.*)?$' % TwitchBaseIE._VALID_URL_BASE
    _PLAYLIST_TYPE = 'profile'

-    _TEST = {
+    _TESTS = [{
        'url': 'http://www.twitch.tv/vanillatv/profile',
        'info_dict': {
            'id': 'vanillatv',
            'title': 'VanillaTV',
        },
        'playlist_mincount': 412,
-    }
+    }, {
+        'url': 'http://m.twitch.tv/vanillatv/profile',
+        'only_matching': True,
+    }]


 class TwitchVideosBaseIE(TwitchPlaylistBaseIE):
@@ -411,14 +430,17 @@ class TwitchAllVideosIE(TwitchVideosBaseIE):
    _PLAYLIST_PATH = TwitchVideosBaseIE._PLAYLIST_PATH + 'archive,upload,highlight'
    _PLAYLIST_TYPE = 'all videos'

-    _TEST = {
+    _TESTS = [{
        'url': 'https://www.twitch.tv/spamfish/videos/all',
        'info_dict': {
            'id': 'spamfish',
            'title': 'Spamfish',
        },
        'playlist_mincount': 869,
-    }
+    }, {
+        'url': 'https://m.twitch.tv/spamfish/videos/all',
+        'only_matching': True,
+    }]


 class TwitchUploadsIE(TwitchVideosBaseIE):
@@ -427,14 +449,17 @@ class TwitchUploadsIE(TwitchVideosBaseIE):
    _PLAYLIST_PATH = TwitchVideosBaseIE._PLAYLIST_PATH + 'upload'
    _PLAYLIST_TYPE = 'uploads'

-    _TEST = {
+    _TESTS = [{
        'url': 'https://www.twitch.tv/spamfish/videos/uploads',
        'info_dict': {
            'id': 'spamfish',
            'title': 'Spamfish',
        },
        'playlist_mincount': 0,
-    }
+    }, {
+        'url': 'https://m.twitch.tv/spamfish/videos/uploads',
+        'only_matching': True,
+    }]


 class TwitchPastBroadcastsIE(TwitchVideosBaseIE):
@@ -443,14 +468,17 @@ class TwitchPastBroadcastsIE(TwitchVideosBaseIE):
    _PLAYLIST_PATH = TwitchVideosBaseIE._PLAYLIST_PATH + 'archive'
    _PLAYLIST_TYPE = 'past broadcasts'

-    _TEST = {
+    _TESTS = [{
        'url': 'https://www.twitch.tv/spamfish/videos/past-broadcasts',
        'info_dict': {
            'id': 'spamfish',
            'title': 'Spamfish',
        },
        'playlist_mincount': 0,
-    }
+    }, {
+        'url': 'https://m.twitch.tv/spamfish/videos/past-broadcasts',
+        'only_matching': True,
+    }]


 class TwitchHighlightsIE(TwitchVideosBaseIE):
@@ -459,14 +487,17 @@ class TwitchHighlightsIE(TwitchVideosBaseIE):
    _PLAYLIST_PATH = TwitchVideosBaseIE._PLAYLIST_PATH + 'highlight'
    _PLAYLIST_TYPE = 'highlights'

-    _TEST = {
+    _TESTS = [{
        'url': 'https://www.twitch.tv/spamfish/videos/highlights',
        'info_dict': {
            'id': 'spamfish',
            'title': 'Spamfish',
        },
        'playlist_mincount': 805,
-    }
+    }, {
+        'url': 'https://m.twitch.tv/spamfish/videos/highlights',
+        'only_matching': True,
+    }]


 class TwitchStreamIE(TwitchBaseIE):
@@ -474,7 +505,7 @@ class TwitchStreamIE(TwitchBaseIE):
    _VALID_URL = r'''(?x)
                    https?://
                        (?:
-                            (?:(?:www|go)\.)?twitch\.tv/|
+                            (?:(?:www|go|m)\.)?twitch\.tv/|
                            player\.twitch\.tv/\?.*?\bchannel=
                        )
                        (?P<id>[^/#?]+)
@@ -508,6 +539,9 @@ class TwitchStreamIE(TwitchBaseIE):
    }, {
        'url': 'https://go.twitch.tv/food',
        'only_matching': True,
+    }, {
+        'url': 'https://m.twitch.tv/food',
+        'only_matching': True,
    }]

    @classmethod
@@ -593,21 +627,23 @@ class TwitchStreamIE(TwitchBaseIE):
        }


-class TwitchClipsIE(InfoExtractor):
+class TwitchClipsIE(TwitchBaseIE):
    IE_NAME = 'twitch:clips'
    _VALID_URL = r'https?://clips\.twitch\.tv/(?:[^/]+/)*(?P<id>[^/?#&]+)'

    _TESTS = [{
-        'url': 'https://clips.twitch.tv/ea/AggressiveCobraPoooound',
+        'url': 'https://clips.twitch.tv/FaintLightGullWholeWheat',
        'md5': '761769e1eafce0ffebfb4089cb3847cd',
        'info_dict': {
-            'id': 'AggressiveCobraPoooound',
+            'id': '42850523',
            'ext': 'mp4',
            'title': 'EA Play 2016 Live from the Novo Theatre',
            'thumbnail': r're:^https?://.*\.jpg',
+            'timestamp': 1465767393,
+            'upload_date': '20160612',
            'creator': 'EA',
            'uploader': 'stereotype_',
-            'uploader_id': 'stereotype_',
+            'uploader_id': '43566419',
        },
    }, {
        # multiple formats
@@ -618,34 +654,63 @@ class TwitchClipsIE(InfoExtractor):
    def _real_extract(self, url):
        video_id = self._match_id(url)

-        webpage = self._download_webpage(url, video_id)
+        status = self._download_json(
+            'https://clips.twitch.tv/api/v2/clips/%s/status' % video_id,
+            video_id)

-        clip = self._parse_json(
-            self._search_regex(
-                r'(?s)clipInfo\s*=\s*({.+?});', webpage, 'clip info'),
-            video_id, transform_source=js_to_json)
+        formats = []

-        title = clip.get('title') or clip.get('channel_title') or self._og_search_title(webpage)
-
-        formats = [{
-            'url': option['source'],
-            'format_id': option.get('quality'),
-            'height': int_or_none(option.get('quality')),
-        } for option in clip.get('quality_options', []) if option.get('source')]
-
-        if not formats:
-            formats = [{
-                'url': clip['clip_video_url'],
-            }]
+        for option in status['quality_options']:
+            if not isinstance(option, dict):
+                continue
+            source = option.get('source')
+            if not source or not isinstance(source, compat_str):
+                continue
+            formats.append({
+                'url': source,
+                'format_id': option.get('quality'),
+                'height': int_or_none(option.get('quality')),
+                'fps': int_or_none(option.get('frame_rate')),
+            })

        self._sort_formats(formats)

-        return {
-            'id': video_id,
-            'title': title,
-            'thumbnail': self._og_search_thumbnail(webpage),
-            'creator': clip.get('broadcaster_display_name') or clip.get('broadcaster_login'),
-            'uploader': clip.get('curator_login'),
-            'uploader_id': clip.get('curator_display_name'),
+        info = {
            'formats': formats,
        }
+
+        clip = self._call_api(
+            'kraken/clips/%s' % video_id, video_id, fatal=False, headers={
+                'Accept': 'application/vnd.twitchtv.v5+json',
+            })
+
+        if clip:
+            quality_key = qualities(('tiny', 'small', 'medium'))
+            thumbnails = []
+            thumbnails_dict = clip.get('thumbnails')
+            if isinstance(thumbnails_dict, dict):
+                for thumbnail_id, thumbnail_url in thumbnails_dict.items():
+                    thumbnails.append({
+                        'id': thumbnail_id,
+                        'url': thumbnail_url,
+                        'preference': quality_key(thumbnail_id),
+                    })
+
+            info.update({
+                'id': clip.get('tracking_id') or video_id,
+                'title': clip.get('title') or video_id,
+                'duration': float_or_none(clip.get('duration')),
+                'views': int_or_none(clip.get('views')),
+                'timestamp': unified_timestamp(clip.get('created_at')),
+                'thumbnails': thumbnails,
+                'creator': try_get(clip, lambda x: x['broadcaster']['display_name'], compat_str),
+                'uploader': try_get(clip, lambda x: x['curator']['display_name'], compat_str),
+                'uploader_id': try_get(clip, lambda x: x['curator']['id'], compat_str),
+            })
+        else:
+            info.update({
+                'title': video_id,
+                'id': video_id,
+            })
+
+        return info
--- a/youtube_dl/extractor/udemy.py
+++ b/youtube_dl/extractor/udemy.py
@@ -18,6 +18,7 @@ from ..utils import (
    int_or_none,
    js_to_json,
    sanitized_Request,
+    try_get,
    unescapeHTML,
    urlencode_postdata,
 )
@@ -58,6 +59,10 @@ class UdemyIE(InfoExtractor):
        # no url in outputs format entry
        'url': 'https://www.udemy.com/learn-web-development-complete-step-by-step-guide-to-success/learn/v4/t/lecture/4125812',
        'only_matching': True,
+    }, {
+        # only outputs rendition
+        'url': 'https://www.udemy.com/how-you-can-help-your-local-community-5-amazing-examples/learn/v4/t/lecture/3225750?start=0',
+        'only_matching': True,
    }]

    def _extract_course_info(self, webpage, video_id):
@@ -101,7 +106,7 @@ class UdemyIE(InfoExtractor):
            % (course_id, lecture_id),
            lecture_id, 'Downloading lecture JSON', query={
                'fields[lecture]': 'title,description,view_html,asset',
-                'fields[asset]': 'asset_type,stream_url,thumbnail_url,download_urls,data',
+                'fields[asset]': 'asset_type,stream_url,thumbnail_url,download_urls,stream_urls,captions,data',
            })

    def _handle_error(self, response):
@@ -115,9 +120,9 @@ class UdemyIE(InfoExtractor):
                error_str += ' - %s' % error_data.get('formErrors')
            raise ExtractorError(error_str, expected=True)

-    def _download_webpage(self, *args, **kwargs):
+    def _download_webpage_handle(self, *args, **kwargs):
        kwargs.setdefault('headers', {})['User-Agent'] = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/603.2.4 (KHTML, like Gecko) Version/10.1.1 Safari/603.2.4'
-        return super(UdemyIE, self)._download_webpage(
+        return super(UdemyIE, self)._download_webpage_handle(
            *args, **compat_kwargs(kwargs))

    def _download_json(self, url_or_request, *args, **kwargs):
@@ -299,9 +304,25 @@ class UdemyIE(InfoExtractor):
                    'url': src,
                })

-        download_urls = asset.get('download_urls')
-        if isinstance(download_urls, dict):
-            extract_formats(download_urls.get('Video'))
+        for url_kind in ('download', 'stream'):
+            urls = asset.get('%s_urls' % url_kind)
+            if isinstance(urls, dict):
+                extract_formats(urls.get('Video'))
+
+        captions = asset.get('captions')
+        if isinstance(captions, list):
+            for cc in captions:
+                if not isinstance(cc, dict):
+                    continue
+                cc_url = cc.get('url')
+                if not cc_url or not isinstance(cc_url, compat_str):
+                    continue
+                lang = try_get(cc, lambda x: x['locale']['locale'], compat_str)
+                sub_dict = (automatic_captions if cc.get('source') == 'auto'
+                            else subtitles)
+                sub_dict.setdefault(lang or 'en', []).append({
+                    'url': cc_url,
+                })

        view_html = lecture.get('view_html')
        if view_html:
@@ -357,6 +378,12 @@ class UdemyIE(InfoExtractor):
                    fatal=False)
                extract_subtitles(text_tracks)

+        if not formats and outputs:
+            for format_id, output in outputs.items():
+                f = extract_output_format(output, format_id)
+                if f.get('url'):
+                    formats.append(f)
+
        self._sort_formats(formats, field_preference=('height', 'width', 'tbr', 'format_id'))

        return {
--- a/youtube_dl/extractor/vimeo.py
+++ b/youtube_dl/extractor/vimeo.py
@@ -16,6 +16,7 @@ from ..utils import (
    ExtractorError,
    InAdvancePagedList,
    int_or_none,
+    merge_dicts,
    NO_DEFAULT,
    RegexNotFoundError,
    sanitized_Request,
@@ -639,16 +640,18 @@ class VimeoIE(VimeoBaseInfoExtractor):
                            'preference': 1,
                        })

-        info_dict = self._parse_config(config, video_id)
-        formats.extend(info_dict['formats'])
+        info_dict_config = self._parse_config(config, video_id)
+        formats.extend(info_dict_config['formats'])
        self._vimeo_sort_formats(formats)

+        json_ld = self._search_json_ld(webpage, video_id, default={})
+
        if not cc_license:
            cc_license = self._search_regex(
                r'<link[^>]+rel=["\']license["\'][^>]+href=(["\'])(?P<license>(?:(?!\1).)+)\1',
                webpage, 'license', default=None, group='license')

-        info_dict.update({
+        info_dict = {
            'id': video_id,
            'formats': formats,
            'timestamp': unified_timestamp(timestamp),
@@ -658,7 +661,9 @@ class VimeoIE(VimeoBaseInfoExtractor):
            'like_count': like_count,
            'comment_count': comment_count,
            'license': cc_license,
-        })
+        }
+
+        info_dict = merge_dicts(info_dict, info_dict_config, json_ld)

        return info_dict

@@ -984,10 +989,10 @@ class VimeoWatchLaterIE(VimeoChannelIE):


 class VimeoLikesIE(InfoExtractor):
-    _VALID_URL = r'https://(?:www\.)?vimeo\.com/user(?P<id>[0-9]+)/likes/?(?:$|[?#]|sort:)'
+    _VALID_URL = r'https://(?:www\.)?vimeo\.com/(?P<id>[^/]+)/likes/?(?:$|[?#]|sort:)'
    IE_NAME = 'vimeo:likes'
    IE_DESC = 'Vimeo user likes'
-    _TEST = {
+    _TESTS = [{
        'url': 'https://vimeo.com/user755559/likes/',
        'playlist_mincount': 293,
        'info_dict': {
@@ -995,7 +1000,10 @@ class VimeoLikesIE(InfoExtractor):
            'description': 'See all the videos urza likes',
            'title': 'Videos urza likes',
        },
-    }
+    }, {
+        'url': 'https://vimeo.com/stormlapse/likes',
+        'only_matching': True,
+    }]

    def _real_extract(self, url):
        user_id = self._match_id(url)
@@ -1004,7 +1012,7 @@ class VimeoLikesIE(InfoExtractor):
            self._search_regex(
                r'''(?x)<li><a\s+href="[^"]+"\s+data-page="([0-9]+)">
                    .*?</a></li>\s*<li\s+class="pagination_next">
-                ''', webpage, 'page count'),
+                ''', webpage, 'page count', default=1),
            'page count', fatal=True)
        PAGE_SIZE = 12
        title = self._html_search_regex(
@@ -1012,7 +1020,7 @@ class VimeoLikesIE(InfoExtractor):
        description = self._html_search_meta('description', webpage)

        def _get_page(idx):
-            page_url = 'https://vimeo.com/user%s/likes/page:%d/sort:date' % (
+            page_url = 'https://vimeo.com/%s/likes/page:%d/sort:date' % (
                user_id, idx + 1)
            webpage = self._download_webpage(
                page_url, user_id,
@@ -1032,7 +1040,7 @@ class VimeoLikesIE(InfoExtractor):

        return {
            '_type': 'playlist',
-            'id': 'user%s_likes' % user_id,
+            'id': '%s_likes' % user_id,
            'title': title,
            'description': description,
            'entries': pl,
--- a/youtube_dl/extractor/vine.py
+++ b/youtube_dl/extractor/vine.py
@@ -2,9 +2,9 @@
 from __future__ import unicode_literals

 import re
-import itertools

 from .common import InfoExtractor
+from ..compat import compat_str
 from ..utils import (
    determine_ext,
    int_or_none,
@@ -112,21 +112,24 @@ class VineIE(InfoExtractor):

 class VineUserIE(InfoExtractor):
    IE_NAME = 'vine:user'
-    _VALID_URL = r'(?:https?://)?vine\.co/(?P<u>u/)?(?P<user>[^/]+)/?(\?.*)?$'
+    _VALID_URL = r'https?://vine\.co/(?P<u>u/)?(?P<user>[^/]+)'
    _VINE_BASE_URL = 'https://vine.co/'
-    _TESTS = [
-        {
-            'url': 'https://vine.co/Visa',
-            'info_dict': {
-                'id': 'Visa',
-            },
-            'playlist_mincount': 46,
+    _TESTS = [{
+        'url': 'https://vine.co/itsruthb',
+        'info_dict': {
+            'id': 'itsruthb',
+            'title': 'Ruth B',
+            'description': '| Instagram/Twitter: itsruthb | still a lost boy from neverland',
        },
-        {
-            'url': 'https://vine.co/u/941705360593584128',
-            'only_matching': True,
-        },
-    ]
+        'playlist_mincount': 611,
+    }, {
+        'url': 'https://vine.co/u/942914934646415360',
+        'only_matching': True,
+    }]
+
+    @classmethod
+    def suitable(cls, url):
+        return False if VineIE.suitable(url) else super(VineUserIE, cls).suitable(url)

    def _real_extract(self, url):
        mobj = re.match(self._VALID_URL, url)
@@ -138,17 +141,14 @@ class VineUserIE(InfoExtractor):
        profile_data = self._download_json(
            profile_url, user, note='Downloading user profile data')

-        user_id = profile_data['data']['userId']
-        timeline_data = []
-        for pagenum in itertools.count(1):
-            timeline_url = '%sapi/timelines/users/%s?page=%s&size=100' % (
-                self._VINE_BASE_URL, user_id, pagenum)
-            timeline_page = self._download_json(
-                timeline_url, user, note='Downloading page %d' % pagenum)
-            timeline_data.extend(timeline_page['data']['records'])
-            if timeline_page['data']['nextPage'] is None:
-                break
-
+        data = profile_data['data']
+        user_id = data.get('userId') or data['userIdStr']
+        profile = self._download_json(
+            'https://archive.vine.co/profiles/%s.json' % user_id, user_id)
        entries = [
-            self.url_result(e['permalinkUrl'], 'Vine') for e in timeline_data]
-        return self.playlist_result(entries, user)
+            self.url_result(
+                'https://vine.co/v/%s' % post_id, ie='Vine', video_id=post_id)
+            for post_id in profile['posts']
+            if post_id and isinstance(post_id, compat_str)]
+        return self.playlist_result(
+            entries, user, profile.get('username'), profile.get('description'))
--- a/youtube_dl/extractor/watchbox.py
+++ b/youtube_dl/extractor/watchbox.py
@@ -69,7 +69,7 @@ class WatchBoxIE(InfoExtractor):

        source = self._parse_json(
            self._search_regex(
-                r'(?s)source\s*:\s*({.+?})\s*,\s*\n', webpage, 'source',
+                r'(?s)source["\']?\s*:\s*({.+?})\s*[,}]', webpage, 'source',
                default='{}'),
            video_id, transform_source=js_to_json, fatal=False) or {}

--- a/youtube_dl/extractor/xiami.py
+++ b/youtube_dl/extractor/xiami.py
@@ -9,8 +9,8 @@ from ..utils import int_or_none
 class XiamiBaseIE(InfoExtractor):
    _API_BASE_URL = 'http://www.xiami.com/song/playlist/cat/json/id'

-    def _download_webpage(self, *args, **kwargs):
-        webpage = super(XiamiBaseIE, self)._download_webpage(*args, **kwargs)
+    def _download_webpage_handle(self, *args, **kwargs):
+        webpage = super(XiamiBaseIE, self)._download_webpage_handle(*args, **kwargs)
        if '>Xiami is currently not available in your country.<' in webpage:
            self.raise_geo_restricted('Xiami is currently not available in your country')
        return webpage
--- a/youtube_dl/extractor/yandexmusic.py
+++ b/youtube_dl/extractor/yandexmusic.py
@@ -34,8 +34,8 @@ class YandexMusicBaseIE(InfoExtractor):
            'youtube-dl with --cookies',
            expected=True)

-    def _download_webpage(self, *args, **kwargs):
-        webpage = super(YandexMusicBaseIE, self)._download_webpage(*args, **kwargs)
+    def _download_webpage_handle(self, *args, **kwargs):
+        webpage = super(YandexMusicBaseIE, self)._download_webpage_handle(*args, **kwargs)
        if 'Нам очень жаль, но&nbsp;запросы, поступившие с&nbsp;вашего IP-адреса, похожи на&nbsp;автоматические.' in webpage:
            self._raise_captcha()
        return webpage
@@ -57,14 +57,14 @@ class YandexMusicTrackIE(YandexMusicBaseIE):
        'info_dict': {
            'id': '4878838',
            'ext': 'mp3',
-            'title': 'Carlo Ambrosio & Fabio Di Bari, Carlo Ambrosio - Gypsy Eyes 1',
+            'title': 'Carlo Ambrosio, Carlo Ambrosio & Fabio Di Bari - Gypsy Eyes 1',
            'filesize': 4628061,
            'duration': 193.04,
            'track': 'Gypsy Eyes 1',
            'album': 'Gypsy Soul',
            'album_artist': 'Carlo Ambrosio',
-            'artist': 'Carlo Ambrosio & Fabio Di Bari, Carlo Ambrosio',
-            'release_year': '2009',
+            'artist': 'Carlo Ambrosio, Carlo Ambrosio & Fabio Di Bari',
+            'release_year': 2009,
        },
        'skip': 'Travis CI servers blocked by YandexMusic',
    }
@@ -120,7 +120,7 @@ class YandexMusicTrackIE(YandexMusicBaseIE):
                track_info.update({
                    'album': album.get('title'),
                    'album_artist': extract_artist(album.get('artists')),
-                    'release_year': compat_str(year) if year else None,
+                    'release_year': int_or_none(year),
                })

        track_artist = extract_artist(track.get('artists'))
--- a/youtube_dl/extractor/youtube.py
+++ b/youtube_dl/extractor/youtube.py
@@ -37,6 +37,7 @@ from ..utils import (
    orderedSet,
    parse_codecs,
    parse_duration,
+    qualities,
    remove_quotes,
    remove_start,
    smuggle_url,
@@ -87,7 +88,7 @@ class YoutubeBaseInfoExtractor(InfoExtractor):
        (username, password) = self._get_login_info()
        # No authentication to be performed
        if username is None:
-            if self._LOGIN_REQUIRED:
+            if self._LOGIN_REQUIRED and self._downloader.params.get('cookiefile') is None:
                raise ExtractorError('No login info available, needed for using %s.' % self.IE_NAME, expected=True)
            return True

@@ -246,9 +247,9 @@ class YoutubeBaseInfoExtractor(InfoExtractor):

        return True

-    def _download_webpage(self, *args, **kwargs):
+    def _download_webpage_handle(self, *args, **kwargs):
        kwargs.setdefault('query', {})['disable_polymer'] = 'true'
-        return super(YoutubeBaseInfoExtractor, self)._download_webpage(
+        return super(YoutubeBaseInfoExtractor, self)._download_webpage_handle(
            *args, **compat_kwargs(kwargs))

    def _real_initialize(self):
@@ -1537,7 +1538,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
            ytplayer_config = self._get_ytplayer_config(video_id, video_webpage)
            if ytplayer_config:
                args = ytplayer_config['args']
-                if args.get('url_encoded_fmt_stream_map'):
+                if args.get('url_encoded_fmt_stream_map') or args.get('hlsvp'):
                    # Convert to the same format returned by compat_parse_qs
                    video_info = dict((k, [v]) for k, v in args.items())
                    add_dash_mpd(video_info)
@@ -1697,9 +1698,11 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
        self.report_information_extraction(video_id)

        # uploader
-        if 'author' not in video_info:
-            raise ExtractorError('Unable to extract uploader name')
-        video_uploader = compat_urllib_parse_unquote_plus(video_info['author'][0])
+        video_uploader = try_get(video_info, lambda x: x['author'][0], compat_str)
+        if video_uploader:
+            video_uploader = compat_urllib_parse_unquote_plus(video_uploader)
+        else:
+            self._downloader.report_warning('unable to extract uploader name')

        # uploader_id
        video_uploader_id = None
@@ -1813,6 +1816,10 @@ class YoutubeIE(YoutubeBaseInfoExtractor):

        chapters = self._extract_chapters(description_original, video_duration)

+        def _extract_filesize(media_url):
+            return int_or_none(self._search_regex(
+                r'\bclen[=/](\d+)', media_url, 'filesize', default=None))
+
        if 'conn' in video_info and video_info['conn'][0].startswith('rtmp'):
            self.report_rtmp_download()
            formats = [{
@@ -1838,6 +1845,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
                                'width': int_or_none(width_height[0]),
                                'height': int_or_none(width_height[1]),
                            }
+            q = qualities(['small', 'medium', 'hd720'])
            formats = []
            for url_data_str in encoded_url_map.split(','):
                url_data = compat_parse_qs(url_data_str)
@@ -1917,13 +1925,19 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
                mobj = re.search(r'^(?P<width>\d+)[xX](?P<height>\d+)$', url_data.get('size', [''])[0])
                width, height = (int(mobj.group('width')), int(mobj.group('height'))) if mobj else (None, None)

+                filesize = int_or_none(url_data.get(
+                    'clen', [None])[0]) or _extract_filesize(url)
+
+                quality = url_data.get('quality_label', [None])[0] or url_data.get('quality', [None])[0]
+
                more_fields = {
-                    'filesize': int_or_none(url_data.get('clen', [None])[0]),
+                    'filesize': filesize,
                    'tbr': float_or_none(url_data.get('bitrate', [None])[0], 1000),
                    'width': width,
                    'height': height,
                    'fps': int_or_none(url_data.get('fps', [None])[0]),
-                    'format_note': url_data.get('quality_label', [None])[0] or url_data.get('quality', [None])[0],
+                    'format_note': quality,
+                    'quality': q(quality),
                }
                for key, value in more_fields.items():
                    if value:
@@ -1969,9 +1983,11 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
                a_format.setdefault('http_headers', {})['Youtubedl-no-compression'] = 'True'
                formats.append(a_format)
        else:
-            unavailable_message = extract_unavailable_message()
-            if unavailable_message:
-                raise ExtractorError(unavailable_message, expected=True)
+            error_message = clean_html(video_info.get('reason', [None])[0])
+            if not error_message:
+                error_message = extract_unavailable_message()
+            if error_message:
+                raise ExtractorError(error_message, expected=True)
            raise ExtractorError('no conn, hlsvp or url_encoded_fmt_stream_map information found in video info')

        # Look for the DASH manifest
@@ -1990,6 +2006,8 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
                    for df in self._extract_mpd_formats(
                            mpd_url, video_id, fatal=dash_mpd_fatal,
                            formats_dict=self._formats):
+                        if not df.get('filesize'):
+                            df['filesize'] = _extract_filesize(df['url'])
                        # Do not overwrite DASH format found in some previous DASH manifest
                        if df['format_id'] not in dash_formats:
                            dash_formats[df['format_id']] = df
@@ -2699,10 +2717,7 @@ class YoutubeFeedsInfoExtractor(YoutubeBaseInfoExtractor):
    def _real_initialize(self):
        self._login()

-    def _real_extract(self, url):
-        page = self._download_webpage(
-            'https://www.youtube.com/feed/%s' % self._FEED_NAME, self._PLAYLIST_TITLE)
-
+    def _entries(self, page):
        # The extraction process is the same as for playlists, but the regex
        # for the video ids doesn't contain an index
        ids = []
@@ -2713,12 +2728,15 @@ class YoutubeFeedsInfoExtractor(YoutubeBaseInfoExtractor):
            # 'recommended' feed has infinite 'load more' and each new portion spins
            # the same videos in (sometimes) slightly different order, so we'll check
            # for unicity and break when portion has no new videos
-            new_ids = filter(lambda video_id: video_id not in ids, orderedSet(matches))
+            new_ids = list(filter(lambda video_id: video_id not in ids, orderedSet(matches)))
            if not new_ids:
                break

            ids.extend(new_ids)

+            for entry in self._ids_to_results(new_ids):
+                yield entry
+
            mobj = re.search(r'data-uix-load-more-href="/?(?P<more>[^"]+)"', more_widget_html)
            if not mobj:
                break
@@ -2730,8 +2748,12 @@ class YoutubeFeedsInfoExtractor(YoutubeBaseInfoExtractor):
            content_html = more['content_html']
            more_widget_html = more['load_more_widget_html']

+    def _real_extract(self, url):
+        page = self._download_webpage(
+            'https://www.youtube.com/feed/%s' % self._FEED_NAME,
+            self._PLAYLIST_TITLE)
        return self.playlist_result(
-            self._ids_to_results(ids), playlist_title=self._PLAYLIST_TITLE)
+            self._entries(page), playlist_title=self._PLAYLIST_TITLE)


 class YoutubeWatchLaterIE(YoutubePlaylistIE):
--- a/youtube_dl/extractor/zattoo.py
+++ b/youtube_dl/extractor/zattoo.py
@@ -0,0 +1,270 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+import re
+from uuid import uuid4
+
+from .common import InfoExtractor
+from ..compat import (
+    compat_HTTPError,
+    compat_str,
+)
+from ..utils import (
+    ExtractorError,
+    int_or_none,
+    try_get,
+    urlencode_postdata,
+)
+
+
+class ZattooBaseIE(InfoExtractor):
+    _NETRC_MACHINE = 'zattoo'
+    _HOST_URL = 'https://zattoo.com'
+
+    _power_guide_hash = None
+
+    def _login(self):
+        (username, password) = self._get_login_info()
+        if not username or not password:
+            self.raise_login_required(
+                'A valid %s account is needed to access this media.'
+                % self._NETRC_MACHINE)
+
+        try:
+            data = self._download_json(
+                '%s/zapi/v2/account/login' % self._HOST_URL, None, 'Logging in',
+                data=urlencode_postdata({
+                    'login': username,
+                    'password': password,
+                    'remember': 'true',
+                }), headers={
+                    'Referer': '%s/login' % self._HOST_URL,
+                    'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
+                })
+        except ExtractorError as e:
+            if isinstance(e.cause, compat_HTTPError) and e.cause.code == 400:
+                raise ExtractorError(
+                    'Unable to login: incorrect username and/or password',
+                    expected=True)
+            raise
+
+        self._power_guide_hash = data['session']['power_guide_hash']
+
+    def _real_initialize(self):
+        webpage = self._download_webpage(
+            self._HOST_URL, None, 'Downloading app token')
+        app_token = self._html_search_regex(
+            r'appToken\s*=\s*(["\'])(?P<token>(?:(?!\1).)+?)\1',
+            webpage, 'app token', group='token')
+        app_version = self._html_search_regex(
+            r'<!--\w+-(.+?)-', webpage, 'app version', default='2.8.2')
+
+        # Will setup appropriate cookies
+        self._request_webpage(
+            '%s/zapi/v2/session/hello' % self._HOST_URL, None,
+            'Opening session', data=urlencode_postdata({
+                'client_app_token': app_token,
+                'uuid': compat_str(uuid4()),
+                'lang': 'en',
+                'app_version': app_version,
+                'format': 'json',
+            }))
+
+        self._login()
+
+    def _extract_cid(self, video_id, channel_name):
+        channel_groups = self._download_json(
+            '%s/zapi/v2/cached/channels/%s' % (self._HOST_URL,
+                                               self._power_guide_hash),
+            video_id, 'Downloading channel list',
+            query={'details': False})['channel_groups']
+        channel_list = []
+        for chgrp in channel_groups:
+            channel_list.extend(chgrp['channels'])
+        try:
+            return next(
+                chan['cid'] for chan in channel_list
+                if chan.get('cid') and (
+                    chan.get('display_alias') == channel_name or
+                    chan.get('cid') == channel_name))
+        except StopIteration:
+            raise ExtractorError('Could not extract channel id')
+
+    def _extract_cid_and_video_info(self, video_id):
+        data = self._download_json(
+            '%s/zapi/program/details' % self._HOST_URL,
+            video_id,
+            'Downloading video information',
+            query={
+                'program_id': video_id,
+                'complete': True
+            })
+
+        p = data['program']
+        cid = p['cid']
+
+        info_dict = {
+            'id': video_id,
+            'title': p.get('title') or p['episode_title'],
+            'description': p.get('description'),
+            'thumbnail': p.get('image_url'),
+            'creator': p.get('channel_name'),
+            'episode': p.get('episode_title'),
+            'episode_number': int_or_none(p.get('episode_number')),
+            'season_number': int_or_none(p.get('season_number')),
+            'release_year': int_or_none(p.get('year')),
+            'categories': try_get(p, lambda x: x['categories'], list),
+        }
+
+        return cid, info_dict
+
+    def _extract_formats(self, cid, video_id, record_id=None, is_live=False):
+        postdata_common = {
+            'https_watch_urls': True,
+        }
+
+        if is_live:
+            postdata_common.update({'timeshift': 10800})
+            url = '%s/zapi/watch/live/%s' % (self._HOST_URL, cid)
+        elif record_id:
+            url = '%s/zapi/watch/recording/%s' % (self._HOST_URL, record_id)
+        else:
+            url = '%s/zapi/watch/recall/%s/%s' % (self._HOST_URL, cid, video_id)
+
+        formats = []
+        for stream_type in ('dash', 'hls', 'hls5', 'hds'):
+            postdata = postdata_common.copy()
+            postdata['stream_type'] = stream_type
+
+            data = self._download_json(
+                url, video_id, 'Downloading %s formats' % stream_type.upper(),
+                data=urlencode_postdata(postdata), fatal=False)
+            if not data:
+                continue
+
+            watch_urls = try_get(
+                data, lambda x: x['stream']['watch_urls'], list)
+            if not watch_urls:
+                continue
+
+            for watch in watch_urls:
+                if not isinstance(watch, dict):
+                    continue
+                watch_url = watch.get('url')
+                if not watch_url or not isinstance(watch_url, compat_str):
+                    continue
+                format_id_list = [stream_type]
+                maxrate = watch.get('maxrate')
+                if maxrate:
+                    format_id_list.append(compat_str(maxrate))
+                audio_channel = watch.get('audio_channel')
+                if audio_channel:
+                    format_id_list.append(compat_str(audio_channel))
+                preference = 1 if audio_channel == 'A' else None
+                format_id = '-'.join(format_id_list)
+                if stream_type in ('dash', 'dash_widevine', 'dash_playready'):
+                    this_formats = self._extract_mpd_formats(
+                        watch_url, video_id, mpd_id=format_id, fatal=False)
+                elif stream_type in ('hls', 'hls5', 'hls5_fairplay'):
+                    this_formats = self._extract_m3u8_formats(
+                        watch_url, video_id, 'mp4',
+                        entry_protocol='m3u8_native', m3u8_id=format_id,
+                        fatal=False)
+                elif stream_type == 'hds':
+                    this_formats = self._extract_f4m_formats(
+                        watch_url, video_id, f4m_id=format_id, fatal=False)
+                elif stream_type == 'smooth_playready':
+                    this_formats = self._extract_ism_formats(
+                        watch_url, video_id, ism_id=format_id, fatal=False)
+                else:
+                    assert False
+                for this_format in this_formats:
+                    this_format['preference'] = preference
+                formats.extend(this_formats)
+        self._sort_formats(formats)
+        return formats
+
+    def _extract_video(self, channel_name, video_id, record_id=None, is_live=False):
+        if is_live:
+            cid = self._extract_cid(video_id, channel_name)
+            info_dict = {
+                'id': channel_name,
+                'title': self._live_title(channel_name),
+                'is_live': True,
+            }
+        else:
+            cid, info_dict = self._extract_cid_and_video_info(video_id)
+        formats = self._extract_formats(
+            cid, video_id, record_id=record_id, is_live=is_live)
+        info_dict['formats'] = formats
+        return info_dict
+
+
+class QuicklineBaseIE(ZattooBaseIE):
+    _NETRC_MACHINE = 'quickline'
+    _HOST_URL = 'https://mobiltv.quickline.com'
+
+
+class QuicklineIE(QuicklineBaseIE):
+    _VALID_URL = r'https?://(?:www\.)?mobiltv\.quickline\.com/watch/(?P<channel>[^/]+)/(?P<id>[0-9]+)'
+
+    _TEST = {
+        'url': 'https://mobiltv.quickline.com/watch/prosieben/130671867-maze-runner-die-auserwaehlten-in-der-brandwueste',
+        'only_matching': True,
+    }
+
+    def _real_extract(self, url):
+        channel_name, video_id = re.match(self._VALID_URL, url).groups()
+        return self._extract_video(channel_name, video_id)
+
+
+class QuicklineLiveIE(QuicklineBaseIE):
+    _VALID_URL = r'https?://(?:www\.)?mobiltv\.quickline\.com/watch/(?P<id>[^/]+)'
+
+    _TEST = {
+        'url': 'https://mobiltv.quickline.com/watch/srf1',
+        'only_matching': True,
+    }
+
+    @classmethod
+    def suitable(cls, url):
+        return False if QuicklineIE.suitable(url) else super(QuicklineLiveIE, cls).suitable(url)
+
+    def _real_extract(self, url):
+        channel_name = video_id = self._match_id(url)
+        return self._extract_video(channel_name, video_id, is_live=True)
+
+
+class ZattooIE(ZattooBaseIE):
+    _VALID_URL = r'https?://(?:www\.)?zattoo\.com/watch/(?P<channel>[^/]+?)/(?P<id>[0-9]+)[^/]+(?:/(?P<recid>[0-9]+))?'
+
+    # Since regular videos are only available for 7 days and recorded videos
+    # are only available for a specific user, we cannot have detailed tests.
+    _TESTS = [{
+        'url': 'https://zattoo.com/watch/prosieben/130671867-maze-runner-die-auserwaehlten-in-der-brandwueste',
+        'only_matching': True,
+    }, {
+        'url': 'https://zattoo.com/watch/srf_zwei/132905652-eishockey-spengler-cup/102791477/1512211800000/1514433500000/92000',
+        'only_matching': True,
+    }]
+
+    def _real_extract(self, url):
+        channel_name, video_id, record_id = re.match(self._VALID_URL, url).groups()
+        return self._extract_video(channel_name, video_id, record_id)
+
+
+class ZattooLiveIE(ZattooBaseIE):
+    _VALID_URL = r'https?://(?:www\.)?zattoo\.com/watch/(?P<id>[^/]+)'
+
+    _TEST = {
+        'url': 'https://zattoo.com/watch/srf1',
+        'only_matching': True,
+    }
+
+    @classmethod
+    def suitable(cls, url):
+        return False if ZattooIE.suitable(url) else super(ZattooLiveIE, cls).suitable(url)
+
+    def _real_extract(self, url):
+        channel_name = video_id = self._match_id(url)
+        return self._extract_video(channel_name, video_id, is_live=True)
--- a/youtube_dl/options.py
+++ b/youtube_dl/options.py
@@ -232,7 +232,7 @@ def parseOpts(overrideArguments=None):
        '--geo-verification-proxy',
        dest='geo_verification_proxy', default=None, metavar='URL',
        help='Use this proxy to verify the IP address for some geo-restricted sites. '
-        'The default proxy specified by --proxy (or none, if the options is not present) is used for the actual downloading.')
+        'The default proxy specified by --proxy (or none, if the option is not present) is used for the actual downloading.')
    geo.add_option(
        '--cn-verification-proxy',
        dest='cn_verification_proxy', default=None, metavar='URL',
@@ -249,6 +249,10 @@ def parseOpts(overrideArguments=None):
        '--geo-bypass-country', metavar='CODE',
        dest='geo_bypass_country', default=None,
        help='Force bypass geographic restriction with explicitly provided two-letter ISO 3166-2 country code (experimental)')
+    geo.add_option(
+        '--geo-bypass-ip-block', metavar='IP_BLOCK',
+        dest='geo_bypass_ip_block', default=None,
+        help='Force bypass geographic restriction with explicitly provided IP block in CIDR notation (experimental)')

    selection = optparse.OptionGroup(parser, 'Video Selection')
    selection.add_option(
--- a/youtube_dl/utils.py
+++ b/youtube_dl/utils.py
@@ -2225,6 +2225,20 @@ def try_get(src, getter, expected_type=None):
                return v


+def merge_dicts(*dicts):
+    merged = {}
+    for a_dict in dicts:
+        for k, v in a_dict.items():
+            if v is None:
+                continue
+            if (k not in merged or
+                    (isinstance(v, compat_str) and v and
+                        isinstance(merged[k], compat_str) and
+                        not merged[k])):
+                merged[k] = v
+    return merged
+
+
 def encode_compat_str(string, encoding=preferredencoding(), errors='strict'):
    return string if isinstance(string, compat_str) else compat_str(string, encoding, errors)

@@ -2574,8 +2588,8 @@ def _match_one(filter_part, dct):
        return op(actual_value, comparison_value)

    UNARY_OPERATORS = {
-        '': lambda v: v is not None,
-        '!': lambda v: v is None,
+        '': lambda v: (v is True) if isinstance(v, bool) else (v is not None),
+        '!': lambda v: (v is False) if isinstance(v, bool) else (v is None),
    }
    operator_rex = re.compile(r'''(?x)\s*
        (?P<op>%s)\s*(?P<key>[a-z_]+)
@@ -3520,10 +3534,13 @@ class GeoUtils(object):
    }

    @classmethod
-    def random_ipv4(cls, code):
-        block = cls._country_ip_map.get(code.upper())
-        if not block:
-            return None
+    def random_ipv4(cls, code_or_block):
+        if len(code_or_block) == 2:
+            block = cls._country_ip_map.get(code_or_block.upper())
+            if not block:
+                return None
+        else:
+            block = code_or_block
        addr, preflen = block.split('/')
        addr_min = compat_struct_unpack('!L', socket.inet_aton(addr))[0]
        addr_max = addr_min | (0xffffffff >> int(preflen))
--- a/youtube_dl/version.py
+++ b/youtube_dl/version.py
@@ -1,3 +1,3 @@
 from __future__ import unicode_literals

-__version__ = '2018.04.03'
+__version__ = '2018.05.18'
Author	SHA1	Message	Date
Sergey M․	7550ea501a	release 2018.05.18	2018-05-18 00:32:51 +07:00
Sergey M․	58197205d3	[ChangeLog] Actualize [ci skip]	2018-05-18 00:30:41 +07:00
Sergey M․	361a965b5c	[vimeo:likes] Relax _VALID_URL and fix single page likes extraction (closes #16475 )	2018-05-17 23:21:40 +07:00
Remita Amine	a3f86160fa	[pluralsight] fix clip id extraction(fixes #16460 )	2018-05-17 13:46:05 +01:00
Remita Amine	1306f5ed72	[mychannels] add support for mychannels.com(closes #15334 )	2018-05-16 19:11:48 +01:00
Remita Amine	58a68d8fda	[moniker] Remove extractor(closes #15336 )	2018-05-16 18:44:33 +01:00
Remita Amine	eea2fafcf5	[pbs] fix embed data extraction(fixes #16474 )	2018-05-16 18:34:25 +01:00
Remita Amine	6843ac5b13	add support for paramountnetwork.com and bellator.com(fixes #15418 )	2018-05-16 17:49:35 +01:00
Remita Amine	54fc90aabf	[youtube] fix hd720 format position	2018-05-16 16:24:44 +01:00
Remita Amine	997530d9d4	[dailymotion] remove fragment part from m3u8 urls(closes #8915 )	2018-05-16 12:04:24 +01:00
Remita Amine	fe3a60f040	[dreisat] improve extraction(closes #15350 ) - extract all formats - extract more format metadata extraction - improve format sorting - use hls native downloader - detect geo-restriction - bypass geo-restriction	2018-05-16 11:30:29 +01:00
Remita Amine	7f34984e81	[dtube] Add new extractor(closes #15201 )	2018-05-16 09:35:47 +01:00
Sergey M․	1e4fe5a7cc	[options] Fix typo (closes #16450 )	2018-05-14 23:42:33 +07:00
Sergey M․	c63ca0eef8	[youtube] Improve format filesize extraction (#16453 )	2018-05-14 23:27:56 +07:00
Sergey M․	84a9fef899	[youtube] Make uploader extraction non fatal (#16444 )	2018-05-13 22:49:01 +07:00
Remita Amine	4c76aa0666	[youtube] fix extraction for embed restricted live streams(fixes #16433 )	2018-05-13 13:20:16 +01:00
Remita Amine	90b633f86b	[nbc] improve info extraction(fixes #16440 )	2018-05-13 11:31:41 +01:00
Sergey M․	07acdc5afc	[twitch:clips] Sort formats	2018-05-12 12:08:54 +07:00
Sergey M․	49fa7de301	[twitch:clips] Fix extraction (closes #16429 )	2018-05-11 23:21:02 +07:00
llyyr	dbd5c502ea	[redditr] Relax _VALID_URL (closes #16426 )	2018-05-10 23:17:23 +07:00
Sergey M․	bc5e4aa57e	[mixcloud] Bypass throttling for HTTP formats (#12579 , #16424 )	2018-05-10 22:22:26 +07:00
Sergey M․	1344d3e169	[nickbr] Relax _VALID_URL (#13230 )	2018-05-10 22:01:13 +07:00
Remita Amine	ff8889cd4d	[teamcoco] fix extraction(closes #16374 )	2018-05-10 08:19:56 +01:00
Sergey M․	9e18bb4c67	release 2018.05.09	2018-05-09 00:36:47 +07:00
Sergey M․	44277998ad	[ChangeLog] Actualize [ci skip]	2018-05-09 00:34:39 +07:00
Sergey M․	05108a496a	[YoutubeDL] Ensure ext exists for automatic captions	2018-05-08 22:57:52 +07:00
Sergey M․	2fbd86352e	[udemy] Extract asset captions	2018-05-08 22:57:01 +07:00
Sergey M․	0ce76801e8	[udemy] Extract stream URLs (closes #16372 )	2018-05-08 22:33:35 +07:00
Sergey M․	789b7774a7	[businessinsider] Add extractor (closes #16387 , closes #16388 , closes #16389 )	2018-05-06 21:58:55 +07:00
Sergey M․	660a230b2d	[cloudflarestream] Add support for cloudflare streams (closes #16375 )	2018-05-05 01:21:52 +07:00
Sergey M․	a90a6b54ee	[watchbox] Fix extraction (closes #16356 )	2018-05-02 20:43:34 +07:00
Remita Amine	3cc0d0b829	[discovery] extract Affiliate/Anonymous Auth Token from cookies(closes #14954 )	2018-05-02 09:32:53 +01:00
Sergey M․	ea1f5e5dbd	[itv:btcc] Add extractor (closes #16139 )	2018-05-02 07:21:24 +07:00
Sergey M․	5f95927a62	Improve geo bypass mechanism * Introduce geo bypass context * Add ability to bypass based on IP blocks in CIDR notation * Introduce --geo-bypass-ip-block	2018-05-02 07:20:59 +07:00
Sergey M․	a93ce61bd5	[tunein] Use live title for live streams (closes #16347 )	2018-05-02 01:29:44 +07:00
Sergey M․	c18142da6e	[itv] Improve extraction (closes #16253 )	2018-05-01 22:48:08 +07:00
Sergey M․	cc42941390	release 2018.05.01	2018-05-01 03:38:57 +07:00
Sergey M․	cc5772c4f0	[ChangeLog] Actualize [ci skip]	2018-05-01 03:30:23 +07:00
Sergey M․	c21692fa94	[kaltura] Improve iframe embeds detection (closes #16337 )	2018-05-01 03:09:04 +07:00
Sergey M․	8513963468	[udemy] Extract outputs renditions (closes #16289 , closes #16291 , closes #16320 , closes #16321 , closes #16334 , closes #16335 )	2018-05-01 02:15:43 +07:00
Sergey M․	67ca1a8ef7	[zattoo] Improve and simplify (closes #14676 )	2018-05-01 01:50:30 +07:00
Alex Seiler	4a73354586	[zattoo] Add extractor (closes #14668 )	2018-05-01 01:50:07 +07:00
Sergey M․	796bf9de45	[yandexmusic] Convert release_year to int	2018-04-29 22:56:07 +07:00
Sergey M․	e5eadfa82f	[udemy,xiami,yandexmusic] Override _download_webpage_handle instead of _download_webpage	2018-04-29 22:54:52 +07:00
Niklas Haas	30226342ab	[youtube] Correctly disable polymer on all requests Rather than just the one that use the _download_webpage helper. The need for this was made apparent by `0fe7783e`, which refactored _download_json in a way that completely avoids the use of _download_webpage, thus breaking youtube. Fixes #16323	2018-04-29 22:35:16 +07:00
Bastian de Groot	01aec84880	[generic] Prefer enclosures over links in RSS feeds	2018-04-29 22:14:37 +07:00
Meneth32	12b0d4e0e1	[redditr] Add support for old.reddit.com URLs	2018-04-29 21:59:40 +07:00
Sergey M․	106c8c3edb	[nrktv] Update API host (closes #16324 )	2018-04-29 19:04:40 +07:00
Sergey M․	500a86a52e	[downloader/fragment] Restart download if .ytdl file is corrupt (closes #16312 )	2018-04-29 00:33:31 +07:00
Sergey M․	7dd6ab4a47	[imdb] Extract all formats (closes #16249 )	2018-04-28 04:51:39 +07:00
Sergey M․	ae1c585cee	[vimeo] Extract JSON LD (closes #16295 )	2018-04-28 02:51:18 +07:00
Sergey M․	e7e4a6e0f9	[extractor/common] Extract interaction statistic	2018-04-28 02:48:03 +07:00
Sergey M․	6cc622327f	[utils] Introduce merge_dicts	2018-04-28 02:47:17 +07:00
Sergey M․	0fe7783ece	[extractor/common] Add _download_json_handle	2018-04-28 01:59:15 +07:00
Sergey M․	c84eae4f66	[funk:channel] Improve extraction (closes #16285 )	2018-04-27 03:45:52 +07:00
Sergey M․	d3711b0050	[devscripts/gh-pages/generate-download.py] Use program checksum from versions.json	2018-04-25 02:14:27 +07:00
Sergey M․	b5802d69f5	release 2018.04.25	2018-04-25 01:12:40 +07:00
Sergey M․	e028d4f506	[ChangeLog] Actualize [ci skip]	2018-04-25 01:07:37 +07:00
Sergey M․	ecb24f7c08	Credit @f2face for #16115	2018-04-25 01:07:32 +07:00
Sergey M․	95284bc281	Credit @TingPing for picarto (#15551 )	2018-04-25 01:07:27 +07:00
Sergey M․	5d0fe6d23e	Credit @Zopieux for #16250	2018-04-25 01:07:23 +07:00
Alexandre Macabies	76030543cd	[openload] Recognize IPv6 stream URLs (closes #16137 )	2018-04-25 00:49:30 +07:00
Sergey M․	0ff51adae6	[twitch] Extract is_live according to status (closes #16259 )	2018-04-24 23:55:06 +07:00
Sergey M․	1cc47c6674	[utils] Fix match_str for boolean meta fields	2018-04-24 23:54:49 +07:00
Sergey M․	99036a1298	[pornflip] Relax _VALID_URL (closes #16258 )	2018-04-23 04:03:11 +07:00
Sergey M․	171625469a	[etonline] Remove extractor (closes #16256 ) Covered by generic extractor	2018-04-23 03:17:34 +07:00
Sergey M․	af751350e8	[Makefile] Add support for pandoc 2 and disable smart extension (closes #16251 ) smart extension rewrites straight quotes as curly quotes, -- as en-dashes and so on that is unwanted behavior.	2018-04-23 02:50:11 +07:00
Sergey M․	2441c1aab1	[breakcom] Fix extraction (closes #16254 )	2018-04-23 00:16:52 +07:00
Sergey M․	70d35d166c	[youtube] Add ability to authenticate with cookies	2018-04-22 06:08:05 +07:00
Sergey M․	3853309fe2	[youtube:feed] Implement lazy playlist extraction (closes #10184 )	2018-04-22 06:07:32 +07:00
Sergey M․	6cdaaf7031	[svt] Improve (closes #15809 )	2018-04-22 05:34:03 +07:00
0x9fff00	488ff2dd3a	[svt] Add support for TV channel live streams (Closes #15279 )	2018-04-22 05:33:40 +07:00
Sergey M․	353f0bde78	[cbssports] PEP 8	2018-04-22 04:57:22 +07:00
Sergey M․	040c6296bb	[ccma] Fix video extraction (closes #15931 )	2018-04-22 04:55:35 +07:00
Sergey M․	a693386df1	[rentv] Improve extraction (closes #15227 )	2018-04-21 23:22:30 +07:00
einstein95	4b8588fe02	[rentv] Fix extraction	2018-04-21 23:22:25 +07:00
Sergey M․	d65a48a0ef	[nick] Add support for nickjr.nl (closes #16230 )	2018-04-20 23:12:13 +07:00
Sergey M․	c194200277	[mofosex] Fix test	2018-04-19 22:38:31 +07:00
Sergey M․	d317973284	[extremetube] Fix metadata extraction	2018-04-19 22:36:33 +07:00
Parmjit Virk	1792bc3a06	[keezmovies] Add support for generic embeds (closes #16134 )	2018-04-19 22:25:51 +07:00
Douglas Su	5a19d231ca	[YoutubeDL] Fix typo in media extension compatibility checker	2018-04-19 22:21:50 +07:00
Remita Amine	d86c5167ae	[nexx] extract new azure urls(closes #16223 )	2018-04-19 15:48:03 +01:00
Remita Amine	b004d9bbf1	[cbssports] fix extraction(fixes #16217 )	2018-04-19 15:08:17 +01:00
Sergey M․	9b3036bd2e	[instagram:user] Fix extraction (closes #16119 )	2018-04-18 10:12:24 +07:00
Sergey M․	e30991f920	[kaltura] Improve embeds detection (closes #16201 )	2018-04-18 01:26:15 +07:00
Dan Salmon	518d5ba519	Fix some tests	2018-04-18 00:10:02 +07:00
Sergey M․	238d42cf5d	[instagram:user] Fix extraction (closes #16119 )	2018-04-17 22:37:50 +07:00
Remita Amine	522d6b5c96	[cbs] skip DRM asset types(fixes #16104 )	2018-04-16 07:48:51 +01:00
Sergey M․	3c92fd1cd5	release 2018.04.16	2018-04-16 01:09:18 +07:00
Sergey M․	bdf7ba6f3a	Set chmod 644 for all extractors	2018-04-16 01:07:21 +07:00
Sergey M․	0e6ccb3905	[ChangeLog] Actualize [ci skip]	2018-04-16 00:56:05 +07:00
Sergey M․	c07cb68e79	[smotri:broadcast] Fix extraction (closes #16180 )	2018-04-16 00:54:21 +07:00
Sergey M․	a42839e548	[picarto] Improve extraction (closes #6205 , closes #12514 , closes #15276 , closes #15551 )	2018-04-16 00:34:47 +07:00
Patrick Griffis	d6166a7602	[picarto] Add extractor	2018-04-16 00:32:15 +07:00
Sergey M․	8e41c9ad01	[vine:user] Improve extraction (closes #16190 )	2018-04-15 22:46:43 +07:00
Timmy	9b5aead6aa	[vine:user] Fix extraction (closes #15514 )	2018-04-15 22:46:30 +07:00
Sergey M․	68ddba20ae	[instagram:user] Remove User-Agent from signature (closes #16119 )	2018-04-13 22:28:33 +07:00
Sergey M․	92ded33a05	[pornhub] Relax _VALID_URLs (closes #16165 )	2018-04-12 04:53:45 +07:00
Sergey M․	64f03e5b4c	[cbc:watch] Re-acquire device token when expired (closes #16160 )	2018-04-11 23:30:19 +07:00
Ray Douglass	d783aee56a	[fxnetworks] Add support for https theplatform URLs (closes #16125 )	2018-04-11 20:11:24 +07:00
Sergey M․	315ab3d500	[instagram:user] Simplify signing (#16119 )	2018-04-11 01:51:57 +07:00
Sergey M․	dd9aea8cbd	[instagram:user] Add request signing (closes #16119 )	2018-04-11 01:25:41 +07:00
Sergey M․	fce7962691	[twitch] Add support for mobile URLs (closes #16146 )	2018-04-10 23:07:37 +07:00
Sergey M․	f7f9757efc	release 2018.04.09	2018-04-09 01:19:27 +07:00
Sergey M․	880ed89d49	[ChangeLog] Actualize [ci skip]	2018-04-09 01:14:47 +07:00
Sergey M․	94c3442e6a	[YoutubeDL] Do not save/restore console title while simulate (closes #16103 )	2018-04-09 01:04:22 +07:00
Sergey M․	069937151e	[generic] Add support for tube8 embeds	2018-04-09 00:37:15 +07:00
Sergey M․	d3431dcb90	[generic] Restrict share-videos.se embeds regex to filter bogus URLs (#16115 )	2018-04-09 00:25:44 +07:00
Surya Oktafendri	1fc37ca3f1	[generic] Add support for share-videos.se embeds (closes #16089 )	2018-04-09 00:19:23 +07:00
Sergey M․	d04ca97616	[odnoklassniki] Improve _VALID_URL readability	2018-04-08 22:21:21 +07:00
GDR!	608c738c7d	[odnoklassniki] Extend _VALID_URL (closes #16081 )	2018-04-08 22:13:00 +07:00
aeph6Ee0	66b686727b	[extractor/common] Relax JSON-LD context check (closes #16006 )	2018-04-08 03:09:42 +07:00
Sergey M․	717ea4e14e	[steam] Bypass mature content check (closes #16113 )	2018-04-08 00:29:43 +07:00
Sergey M․	cae5d9705c	[acast] Extract more metadata	2018-04-08 00:21:55 +07:00
Sergey M․	1c9b1a4494	[acast] Fix extraction (closes #16118 )	2018-04-08 00:08:45 +07:00
Sergey M․	ff826177cc	[instagram:user] Fix extraction (closes #16119 )	2018-04-07 23:58:57 +07:00
Parmjit Virk	9d15be3a5b	[drtuber] Fix title extraction (closes #16107 )	2018-04-07 21:39:21 +07:00
Sergey M․	e2750e1437	[liveleak] Extend _VALID_URL (closes #16117 )	2018-04-07 20:55:01 +07:00
Sergey M․	e944737c59	[openload] Add support for oload.xyz	2018-04-06 23:40:15 +07:00
Sergey M․	fdfb32a0dd	[openload] Relax stream URL regex	2018-04-06 00:15:22 +07:00
Sergey M․	235d828b7b	[openload] Fix extraction (closes #16099 )	2018-04-05 23:49:15 +07:00
Sergey M․	1236ac6b0b	[svtplay] Share svtplay regex	2018-04-05 00:29:13 +07:00
Sergey M․	df146eb282	[svtplay:series] Add support for season URLs	2018-04-05 00:29:08 +07:00
Sergey M․	b71bb3ba8b	[svtplay:series] Improve extraction (closes #16059 )	2018-04-05 00:29:02 +07:00
Mattias Wadman	fd97fa7bfc	[svtplay:series] Add extractor Related to #11130	2018-04-05 00:28:58 +07:00