Compare commits

...

3614 Commits

Author SHA1 Message Date
2bb70750a9 release 2020.12.02 2020-12-02 01:37:40 +07:00
09d923f2eb [ChangeLog] Actualize
[ci skip]
2020-12-02 01:22:43 +07:00
37d979ad33 [tva] Add support for qub.ca (closes #27235) 2020-12-01 12:25:02 +01:00
95ac4de229 [toggle] Detect DRM protected videos (closes #16479)(closes #20805) 2020-12-01 10:38:53 +01:00
d3e142b3fa [toggle] Add support for new MeWatch URLs (closes #27256) 2020-12-01 10:13:06 +01:00
132aece1ed [youtube:tab] Extract channels only from channels tab (closes #27266) 2020-12-01 04:44:10 +07:00
3e4e338133 [cspan] Extract info from jwplayer data (closes #3672, closes #3734, closes #10638, closes #13030, closes #18806, closes #23148, closes #24461, closes #26171, closes #26800, closes #27263) 2020-12-01 01:53:12 +07:00
be19ae11fd [cspan] Pass Referer header with format's video URL (#26032) (closes #25729) 2020-12-01 01:14:29 +07:00
59d63d8d4a [youtube] Improve age-gated videos extraction (closes #27259) 2020-12-01 00:49:26 +07:00
cfeba5d17f [mediaset] add support for movie URLs(closes #27240) 2020-11-30 09:46:15 +01:00
6da0e5e7a2 [yandexmusic] Refactor and add support for artist's tracks and albums (closes #11887, closes #22284) 2020-11-30 00:25:06 +07:00
d6ce649f15 [yandexmusic:track] Fix extraction (closes #26449, closes #26669, closes #26747, closes #26748, closes #26762) 2020-11-29 22:15:51 +07:00
b449b73dcc release 2020.11.29 2020-11-29 13:53:01 +07:00
16c822e91e [ChangeLog] Actualize
[ci skip]
2020-11-29 13:49:12 +07:00
4318170779 [drtv] Extend _VALID_URL (#27243) 2020-11-29 13:44:36 +07:00
fb626c0586 [tiktok] Fix extraction (closes #20809, closes #22838, closes #22850, closes #25987, closes #26281, closes #26411, closes #26639, closes #26776, closes #27237) 2020-11-29 08:09:20 +07:00
717d1d2d5a [ina] Add support for mobile URLs (#27229) 2020-11-29 04:15:53 +07:00
9585b376db [YoutubeDL] Write static debug to stderr and respect quiet for dynamic debug (closes #14579, closes #22593)
TODO: logging and verbosity needs major refactoring (refs #10894)
2020-11-29 04:04:06 +07:00
f04cfe24e0 [pornhub] Fix like and dislike count extraction (closes #27227) (#27234) 2020-11-29 02:32:13 +07:00
20c50c6556 [youtube] Improve yt initial player response extraction (closes #27216) 2020-11-28 15:02:31 +07:00
f9f9699f2f [videa] improve extraction 2020-11-26 12:56:49 +01:00
a3cf22e590 [videa] Adapt to updates (#26301)
closes #25973, closes #25650.
2020-11-26 11:55:06 +00:00
99de2f38d3 [spreaker] fix SpreakerShowIE test URL 2020-11-25 21:39:17 +01:00
9fe50837c3 release 2020.11.26 2020-11-26 03:05:51 +07:00
4dc545553f [ChangeLog] Actualize
[ci skip]
2020-11-26 03:03:51 +07:00
686e898fde [spreaker] Add extractor (closes #13480, closes #13877) 2020-11-26 02:58:48 +07:00
3a78198a96 [vlive] improve extraction for geo-restricted videos 2020-11-25 11:40:37 +01:00
836c810716 [vlive] Add support for post URLs(closes #27122)(closes #27123) 2020-11-25 11:26:26 +01:00
97c5be383c [viki] fix video API request(closes #27184) 2020-11-24 23:25:03 +01:00
a7ea88537a [bbc] fix BBC Three clip extraction 2020-11-24 22:54:08 +01:00
6c35de4c6b [bbc] fix BBC News videos extraction 2020-11-24 22:49:04 +01:00
579d43951d [medaltv] improve extraction 2020-11-24 18:29:46 +01:00
e24ebeafd1 [medaltv] Add new extractor (#27149) 2020-11-24 18:27:33 +01:00
01c92973dd [youtube] imporve music metadata and license extraction(closes #26013) 2020-11-23 22:44:59 +01:00
f4415faa46 [downloader/fragment] Set final file's mtime according to last fragment's Last-Modified header (closes #11718, closes #18384, closes #27138) 2020-11-24 04:17:24 +07:00
a86ce9d7a1 [nrk] Fix extraction 2020-11-24 02:59:49 +07:00
37258c644f [cda] Fix extraction (closes #17803, closes #24458, closes #24518, closes #26381) 2020-11-24 02:18:40 +07:00
d0512ac4c5 release 2020.11.24 2020-11-24 00:23:14 +07:00
da4eaa15a2 [ChangeLog] Actualize
[ci skip]
2020-11-24 00:20:42 +07:00
191286265d [youtube:tab] Fix feeds extraction (closes #25695, closes #26452) 2020-11-24 00:10:25 +07:00
3234272818 [youtube:tab] PEP 8 2020-11-23 22:34:27 +07:00
9d2c90354f [youtube:favorites] Restore extractor 2020-11-23 22:33:21 +07:00
316b10855a [youtube:tab] Fix some weird typo (closes #27157) 2020-11-23 22:19:25 +07:00
484fe78737 [pinterest] Add support for large collections(more than 25 pins) 2020-11-23 14:16:38 +01:00
2cd43a00d1 [franceinter] flake8 2020-11-22 19:38:45 +01:00
dd0f524c69 [franceinter] add thumbnail url (#27153)
Co-authored-by: remitamine <remitamine@gmail.com>
2020-11-22 19:35:53 +01:00
c84f9475b8 [box] Add new extractor(#5949) 2020-11-22 17:39:41 +01:00
15f2734791 [nytimes] Add new cooking.nytimes.com extractor (#27143)
* [nytimes] support cooking.nytimes.com, resolves #27112

Co-authored-by: remitamine <remitamine@gmail.com>
2020-11-22 14:12:47 +01:00
cb6e24f946 [lbry] relax _VALID_URL regex(closes #27144) 2020-11-22 13:16:03 +01:00
9d531aa291 [rumble] add support for embed pages(#10785) 2020-11-22 13:07:04 +01:00
e9cbb98a0f [skyit] add support for multiple Sky Italia websites(closes #26629) 2020-11-22 13:01:56 +01:00
193422e12a [extractor/common] add generic support for akamai http format extraction 2020-11-22 12:54:55 +01:00
c4cabf040e [pinterest] Add extractor (closes #25747) 2020-11-22 05:04:01 +07:00
f23eceebbf release 2020.11.21.1 2020-11-21 23:59:11 +07:00
650bd8f623 [ChangeLog] Actualize
[ci skip]
2020-11-21 23:56:32 +07:00
5d8cb4367d release 2020.11.21 2020-11-21 23:54:33 +07:00
82abc13aed [youtube:tab] Comment out test 2020-11-21 23:41:49 +07:00
1e72660c9b [svtplay] Fix test title 2020-11-21 23:36:25 +07:00
049f224248 [svtplay] Add support for svt.se/barnkanalen (closes #24817) 2020-11-21 23:35:32 +07:00
0ada1b90b8 [svt] Extract timestamp and thumbnail in more cases (#27130)
Add timestamp, set to "valid from" which i think could been seen as publish time.
Add thumbnail in more cases, seems to was only done in the embedded data case for some reason.
Switch svtplay test url to an existing video and also one with no expire date.
Also add an additional thumbnail url test regex.
2020-11-21 23:24:37 +07:00
71ddc222ad Fix typos (#27084)
* spelling: authorization

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: brightcove

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: creation

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: exceeded

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: exception

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: extension

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: extracting

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: extraction

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: frontline

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: improve

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: length

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: listsubtitles

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: multimedia

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: obfuscated

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: partitioning

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: playlist

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: playlists

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: restriction

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: services

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: split

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: srmediathek

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: support

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: thumbnail

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: verification

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: whitespaces

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-11-21 22:00:05 +07:00
21292c0649 [youtube] Fix error reason extraction (#27081) 2020-11-21 21:52:20 +07:00
46a265a2da [youtube] Fix like and dislike count extraction (closes #25977) 2020-11-21 20:49:56 +07:00
e2096776b9 [youtube:tab] Add support for current video and fix lives extraction (closes #27126) 2020-11-21 20:48:13 +07:00
a1c88c4819 [infoq] fix format extraction(closes #25984) 2020-11-20 23:23:55 +01:00
51ca93d751 [francetv] Update to fix thumbnail URL issue (#27120)
Fix the thumbnail URL. The issue was here for many years, never fixed. It's done ! :-)

Example : https://www.france.tv/france-2/de-gaulle-l-eclat-et-le-secret/de-gaulle-l-eclat-et-le-secret-saison-1/2035247-solitude.html

failed thumbnail url generated : http://pluzz.francetv.fr/staticftv/ref_emissions/2020-11-02/EMI_1104da66f533cc7dc5d0d07a181a18c2e2fe1d81_20201014122553940.jpg

right thumbnail url fixed : https://sivideo.webservices.francetelevisions.fr/staticftv/ref_emissions/2020-11-02/EMI_1104da66f533cc7dc5d0d07a181a18c2e2fe1d81_20201014122553940.jpg
2020-11-20 22:44:08 +01:00
a7e0531999 [downloader/http] Fix crash during urlopen caused by missing reason of URLError 2020-11-21 04:22:21 +07:00
ab0eda99e1 [YoutubeDL] Fix --ignore-errors for playlists with generator-based entries of url_transparent (closes #27064) 2020-11-21 04:00:08 +07:00
ec99f47108 [youtube:tab] Replace some test URLs with RIAA-friendly ones 2020-11-20 23:34:46 +07:00
b31b5f4434 [youtube] Improve yt initial data extraction (closes #27093) 2020-11-20 23:21:52 +07:00
86f2fa1590 [discoverynetworks] add support new TLC/DMAX URLs(closes #27100) 2020-11-20 10:47:52 +01:00
7bc7fbce23 [rai] fix protocol relative relinker URLs(closes #22766) 2020-11-20 10:26:55 +01:00
a78e530c14 [rai] fix unavailable video format detection 2020-11-20 10:10:57 +01:00
af7bb684c0 [rai] improve extraction 2020-11-20 10:01:56 +01:00
dd9e0f58f3 [rai] Fix extraction for recent raiplay.it updates (#27077)
- Remove first test of RaiPlayIE: it is no longer available
- Make RaiPlayIE extension-agnostic (passing possible `.json' URLs is now
  supported too)
- Adjust RaiPlayLiveIE to recent raiplay.it updates.  Passing it as
  `url_transparent' is no longer supported (there is no longer an accessible
  ContentItem)
- Adjust RaiPlayPlaylistIE to recent raiplay.it updates and instruct it about
  ContentSet-s.
- Update a RaiIE test and remove two tests that are no longer availables

Thanks to @remitamine for the review!
2020-11-20 10:00:05 +01:00
59e583f7e8 [viki] improve format extraction 2020-11-19 22:49:28 +01:00
daa25d4142 [viki] fix stream extraction from mpd (#27092)
Co-authored-by: beefchop <beefchop@users.noreply.github.com>
2020-11-19 21:38:09 +01:00
25a35cb38a [googledrive] fix format extraction(closes #26979) 2020-11-19 20:01:24 +01:00
2cf8003638 [amara] improve extraction 2020-11-19 17:29:30 +01:00
cf1a8668e8 [Amara] Add new extractor (#20618)
* [Amara] Add new extractor
2020-11-19 17:26:53 +01:00
2dbb45ae82 [vimeo:album] fix extraction(closes #27079) 2020-11-19 13:13:33 +01:00
4fe190df70 [mtv] fix mgid extraction(closes #26841) 2020-11-19 11:57:09 +01:00
039e715b30 release 2020.11.19 2020-11-19 05:22:27 +07:00
32152bab7a [ChangeLog] Actualize
[ci skip]
2020-11-19 05:21:09 +07:00
b1347a5881 [youporn] Fix upload date extraction and make comment count optional (closes #26986) 2020-11-19 05:16:25 +07:00
91e954587f [arte] Rework extractors
* Reimplement embed and playlist extractors to delegate to the single entrypoint artetv extractor
  Beware reluctant download archive extractor keys breakage.
* Improve embeds detection (closes #27057)
- Remove obsolete code
2020-11-19 05:02:04 +07:00
9b505185da [arte] Extract m3u8 formats (closes #27061) 2020-11-19 03:26:49 +07:00
444a68e0ec [mgtv] fix format extraction(closes #26415) 2020-11-18 21:07:00 +01:00
c7178f0f7a [extractor/common] Output error for invalid URLs in _is_valid_url (refs #21400, refs #24151, refs #25617, refs #25618, refs #25586, refs #26068, refs #27072) 2020-11-18 23:31:35 +07:00
5c3f7014ef [lbry] add support for odysee.com domain(closes #26806) 2020-11-18 00:41:07 +01:00
d65628ef03 [francetv] improve info extraction 2020-11-17 23:16:04 +01:00
8a6c5b0806 [francetv] Add fallback video url extraction (#27047)
Fallback on another API endpoint when no video formats are found.

Closes ytdl-org#22561
2020-11-17 23:06:19 +01:00
9360936f26 release 2020.11.18 2020-11-18 04:15:30 +07:00
bb2b89e077 [ChangeLog] Actualize
[ci skip]
2020-11-18 04:11:58 +07:00
2d7a29081c [spiegel] Fix extraction (closes #24206, closes #24767)
Code picked from PR #24767 since original repo is not available due to takedown.
2020-11-18 04:09:02 +07:00
2864179293 [youtube] Improve extraction
+ Add support for --no-playlist (closes #27009)
* Improve playlist and mix extraction (closes #26390, closes #26509, closes #26534, closes #27011)
+ Extract playlist uploader data
* Update tests
2020-11-18 03:34:08 +07:00
284f8306df [youtube:tab] fix view_count extraction(closes #27051) 2020-11-17 20:32:50 +01:00
aa613ef7e1 [malltv] fix extraction(closes #27035) 2020-11-17 19:13:38 +01:00
cb2b9a22a5 [bandcamp] extract playlist_description(closes #22684) 2020-11-17 14:46:02 +01:00
5b867c15a8 [urplay] fix extraction(closes #26828) 2020-11-17 13:12:20 +01:00
f8c749f12c [youtube:tab] Fix playlist title extraction (closes #27015) 2020-11-17 07:01:41 +07:00
7d509c613b [youtube] Fix chapters extraction (closes #26005) 2020-11-17 04:28:35 +07:00
b92e95aa01 release 2020.11.17 2020-11-17 03:59:54 +07:00
e3cad6bd99 [ChangeLog] Actualize
[ci skip]
2020-11-17 03:50:12 +07:00
6699b6ce41 [youtube:tab] Fix extraction with cookies provided (closes #27005) 2020-11-17 03:40:57 +07:00
91dcde8a38 [lrt] fix extraction with empty tags(closes #20264) 2020-11-16 21:28:12 +01:00
11f3471c4b [ndr:embed:base] Extract subtitles (closes #25447, closes #26106) 2020-11-17 03:24:23 +07:00
f4093b34f6 [servus] Add support for pm-wissen.com (closes #25869) 2020-11-17 03:02:07 +07:00
a80b23c373 [servus] Fix extraction (closes #26872, closes #26967, closes #26983, closes #27000) 2020-11-17 02:57:26 +07:00
2e7fa18bb9 [xtube] Fix extraction (closes #26996) 2020-11-17 01:32:48 +07:00
fe07e788bf [utils] Skip ! prefixed code in js_to_json 2020-11-17 01:30:43 +07:00
6d3bdcf217 [lrt] fix extraction 2020-11-16 19:17:10 +01:00
ec2a2ab441 [lbry] Add new extractor 2020-11-16 19:04:55 +01:00
9448a20312 [condenast] fix extraction and extract subtitles 2020-11-16 18:57:56 +01:00
3f1748b944 [bandcamp] fix extraction 2020-11-16 17:26:09 +01:00
fe13087cd1 [rai] fix RaiPlay extraction 2020-11-16 17:17:52 +01:00
f22fa82d7f [extractors] Remove VLivePlaylistIE import 2020-11-16 16:36:56 +01:00
059fa9aa81 [vlive] fix extraction 2020-11-16 16:35:08 +01:00
650aec4a98 [usanetwork] fix extraction 2020-11-16 16:21:48 +01:00
2ea9c97432 [nbc] fix NBCNews/Today/MSNBC extraction 2020-11-16 16:18:37 +01:00
efc589b865 [devscripts/make_lazy_extractors] Correct a spelling mistake (#26991) 2020-11-16 22:08:20 +07:00
1737ea69b9 [cnbc] fix extraction 2020-11-16 15:55:07 +01:00
1fb034d029 [youtube] Remove RIAA copyrighted media from tests as per [1]
1. Github dmca and 1201 notice re youtube-dl 9-21-20
2020-11-16 21:08:16 +07:00
28f9568a84 release 2020.11.12 2020-11-12 06:23:46 +07:00
5bbdadd5f8 [ChangeLog] Actualize
[ci skip]
2020-11-12 06:18:16 +07:00
2de2ca6659 [youtube] Rework extractors
WIP
2020-11-12 06:16:37 +07:00
051071203c release 2020.11.01.1 2020-11-01 08:58:40 +07:00
34299510bb release 2020.11.01 2020-11-01 08:52:27 +07:00
b9bceba37c [ChangeLog] Actualize
[ci skip]
2020-11-01 07:34:20 +07:00
6d4733ce7b [youtube] Fix JS player URL extraction 2020-11-01 06:52:00 +07:00
416da574ec [ytsearch] Fix extraction (closes #26920) 2020-10-23 21:31:37 +07:00
48c5663c5f [afreecatv] Fix typo (#26970) 2020-10-22 19:15:05 +07:00
7d740e7dc7 [23video] Relax _VALID_URL (#26870) 2020-10-20 00:56:23 +07:00
4eda10499e [utils] Don't attempt to coerce JS strings to numbers in js_to_json (#26851)
The current logic in `js_to_json` tries to rewrite octal/hex numbers to
decimal. However, when the logic actually happens the `"` or `'` have
already been trimmed off. This causes what were originally strings, that
happen to look like octal/hex numbers, to get rewritten to decimal and
returned as a number rather than a string.

In practive something like:

```js
{
  "0x40": "foo",
  "040": "bar",
}
```

would get rewritten as:

```json
{
  64: "foo",
  32: "bar
}
```

This is problematic since this isn't valid JSON as you cannot have
non-string keys.
2020-10-18 00:10:41 +07:00
605535776a [ustream] Add support for video.ibm.com (#26894) 2020-10-17 23:14:46 +07:00
1050e0d09f [iqiyi] Fix typo (#26884) 2020-10-17 23:02:17 +07:00
d65d89183f [expressen] Add support for di.se (closes #26670) 2020-09-24 07:37:10 +07:00
0c92f1e96b [iprima] Improve video id extraction (#26507) (closes #26494) 2020-09-24 06:46:58 +07:00
adae9e844b [README.md] Fix autonumber sequence description (refs #26686) 2020-09-24 06:36:07 +07:00
c5764b3f89 [downloader/http] Properly handle missing message in SSLError (closes #26646) 2020-09-22 07:01:59 +07:00
0837992a22 [downloader/http] Fix access to not yet opened stream in retry 2020-09-22 06:44:14 +07:00
b55715934b release 2020.09.20 2020-09-20 12:30:45 +07:00
bbc3b5b4bb [ChangeLog] Actualize
[ci skip]
2020-09-20 12:24:32 +07:00
1ca5f821c8 [redtube] Extend _VALID_URL (#26506) 2020-09-20 11:39:42 +07:00
defc820b70 [twitch] Switch streams to GraphQL and refactor (closes #26535) 2020-09-20 10:05:00 +07:00
82ef02e936 [telequebec] Fix issues (closes #26368) 2020-09-19 07:56:00 +07:00
b856b3997c [telequebec] Add support for brightcove videos (closes #25833) 2020-09-19 07:52:57 +07:00
cd85a1bb8b [pornhub] Extract metadata from JSON-LD (closes #26614) 2020-09-19 06:34:34 +07:00
ce5b904050 [extractor/common] Relax interaction count extraction in _json_ld 2020-09-19 06:33:17 +07:00
ad06b99dd4 [extractor/common] Extract author as uploader for VideoObject in _json_ld 2020-09-19 06:13:42 +07:00
540b9f5164 [pornhub] Fix view count extraction (#26621) (refs #26614) 2020-09-19 05:59:19 +07:00
6e65a2a67e [downloader/hls] Fix incorrect end byte in Range HTTP header for media segments with EXT-X-BYTERANGE (#24512) (closes #14748)
The end of the byte range is the first byte that is NOT part of the to
be downloaded range. So don't include it into the requested HTTP
download range, as this additional byte leads to a broken TS packet and
subsequently to e.g. visible video corruption.

Fixes #14748.
2020-09-18 05:26:56 +07:00
f8c7bed133 [extractor/common] Handle ssl.CertificateError in _request_webpage (closes #26601)
ssl.CertificateError is raised on some python versions <= 3.7.x
2020-09-18 03:41:16 +07:00
cdc55e666f [downloader/http] Improve timeout detection when reading block of data (refs #10935) 2020-09-18 03:32:54 +07:00
86b7c00adc [downloader/http] Retry download when urlopen times out (#26603) (refs #10935) 2020-09-18 03:15:44 +07:00
e8c5d40bc8 release 2020.09.14 2020-09-14 03:37:36 +07:00
ca7ebc4e5e [ChangeLog] Actualize
[ci skip]
2020-09-14 03:35:18 +07:00
bff857a8af [postprocessor/embedthumbnail] Fix issues (closes #25717)
* Fix WebP with wrong extension processing
* Fix embedding of thumbnails with % character in path
2020-09-14 03:28:31 +07:00
a31a022efd [postprocessor/embedthumbnail] Add support for non jpeg/png thumbnails (closes #25687) 2020-09-14 03:10:01 +07:00
45f6362464 [rtlnl] Extend _VALID_URL for new embed URL schema 2020-09-13 21:42:06 +07:00
97f34a48d7 [rtlnl] Extend _VALID_URL (#26549) (closes #25821) 2020-09-13 21:38:16 +07:00
ea74e00b3a [youtube] Fix empty description extraction (#26575) (closes #26006) 2020-09-13 21:23:21 +07:00
06cd4cdb25 [srgssr] Extend _VALID_URL (closes #26555, closes #26556, closes #26578) 2020-09-13 21:07:25 +07:00
da2069fb22 [googledrive] Use redirect URLs for source format (closes #18877, closes #23919, closes #24689, closes #26565) 2020-09-13 20:49:32 +07:00
95c9810015 [svtplay] Fix id extraction (closes #26576) 2020-09-13 18:59:37 +07:00
b03eebdb6a [redbulltv] improve support for rebull.com TV localized URLS(#22063) 2020-09-13 11:26:11 +01:00
1f7675451c [redbulltv] Add support for new redbull.com TV URLs(closes #22037)(closes #22063) 2020-09-12 19:27:58 +01:00
aa27253556 [soundcloud] Reduce pagination limit to fix 502 Bad Gateway errors when listing a user's tracks. (#26557)
Per the documentation here https://developers.soundcloud.com/blog/offset-pagination-deprecated the maximum limit is 200, so let's respect that (even if a higher value sometimes works).

Co-authored-by: tfvlrue <tfvlrue>
2020-09-12 09:35:11 +00:00
d51e23d9fc release 2020.09.06 2020-09-06 13:00:41 +07:00
6cd452acff [ChangeLog] Actualize
[ci skip]
2020-09-06 12:57:56 +07:00
50e9fcc1fd [nrktv:episode] Improve video id extraction (closes #25594, closes #26369, closes #26409) 2020-09-06 12:43:50 +07:00
16ee69c1b7 [youtube] Fix age gate content detection (#26100) (closes #26152, closes #26311, closes #26384) 2020-09-06 11:44:53 +07:00
67171ed7e9 [youtube:user] Extend _VALID_URL (closes #26443) 2020-09-06 11:31:28 +07:00
1d9bf655e6 [utils] Recognize wav mimetype (closes #26463) 2020-09-06 11:19:53 +07:00
62ae19ff76 [xhamster] Improve initials regex (#26526) (closes #26353) 2020-09-06 11:10:27 +07:00
5ed05f26ad [svtplay] Fix svt id extraction (closes #26425, closes #26428, closes #26438) 2020-09-06 10:45:57 +07:00
841b683804 [twitch] Rework extractors (closes #12297, closes #20414, closes #20604, closes #21811, closes #21812, closes #22979, closes #24263, closes #25010, closes #25553, closes #25606)
* Switch to GraphQL.
+ Add support for collections.
+ Add support for clips and collections playlists.
2020-09-06 10:45:34 +07:00
f5863a3ea0 [biqle] improve video_ext extraction 2020-08-27 19:20:41 +01:00
10709fc7c6 [xhamster] Extend _VALID_URL (closes #25927) 2020-08-12 21:51:50 +07:00
a7e348556a [xhamster] Fix extraction (closes #26157) (#26254) 2020-08-12 21:42:17 +07:00
6cb30ea5ed [xhamster] Extend _VALID_URL (closes #25789) (#25804) 2020-08-12 21:37:22 +07:00
a4ed50bb84 release 2020.07.28 2020-07-28 05:13:03 +07:00
570611955f [ChangeLog] Actualize
[ci skip]
2020-07-28 05:07:54 +07:00
e450f6cb63 [youtube] Fix sigfunc name extraction (closes #26134, closes #26135, closes #26136, closes #26137) 2020-07-28 05:05:38 +07:00
a115e07594 [youtube] Improve description extraction (closes #25937) (#25980) 2020-07-14 12:01:15 +01:00
718393c632 [wistia] Restrict embed regex (closes #25969) 2020-07-11 18:27:19 +07:00
07af16b92e [youtube] Prevent excess HTTP 301 (#25786) 2020-07-01 02:56:16 +07:00
e942cfd1a7 [youtube:playlists] Extend _VALID_URL (closes #25810) 2020-06-28 10:30:03 +07:00
9a7e5cb88a [bellmedia] add support for cp24.com clip URLs(closes #25764) 2020-06-23 15:09:13 +01:00
2391941f28 [brightcove] Improve embed detection (closes #25674) 2020-06-16 17:38:25 +07:00
9ff6165a81 release 2020.06.16.1 2020-06-16 06:22:01 +07:00
1c748722f9 [ChangeLog] Actualize
[ci skip]
2020-06-16 06:19:23 +07:00
ee0b726cd7 [youtube] Force old layout (closes #25682, closes #25683, closes #25680, closes #25686) 2020-06-16 06:17:53 +07:00
dbeafce5d5 [youtube] Fix categories and improve tags extraction 2020-06-16 03:13:39 +07:00
ed604ce7bc release 2020.06.16 2020-06-16 02:53:33 +07:00
7adc7ca547 [ChangeLog] Actualize
[ci skip]
2020-06-16 02:52:09 +07:00
a6211d237b [youtube] Fix uploader id and uploader URL extraction 2020-06-16 02:43:09 +07:00
7b16239a49 [youtube] Improve view count extraction 2020-06-16 02:38:45 +07:00
37357d21a9 [youtube] Fix upload date extraction 2020-06-16 02:37:19 +07:00
b477fc1314 [youtube] Fix thumbnails extraction and remove uploader id extraction warning (closes #25676) 2020-06-16 02:29:04 +07:00
d84b21b427 [youtube] Fix playlist and feed extraction (closes #25675) 2020-06-16 02:01:12 +07:00
48bd042ce7 [facebook] Support single-video ID links
I stumbled upon this at https://www.facebook.com/bwfbadminton/posts/10157127020046316 . No idea how prevalent it is yet.
2020-06-14 13:17:51 +02:00
84213ea8d4 [youtube] Extract chapters from JSON (closes #24819) 2020-06-06 04:22:10 +07:00
562de77f41 [kaltura] Add support for multiple embeds on a webpage (closes #25523) 2020-06-06 02:14:35 +07:00
e1723c4bac release 2020.06.06 2020-06-06 01:51:39 +07:00
607d204551 [ChangeLog] Actualize
[ci skip]
2020-06-06 01:49:27 +07:00
a5b6102ea8 [tele5] Bypass geo restriction 2020-06-06 01:45:05 +07:00
b77888228d [jwplatform] Add support for bypass geo restriction 2020-06-06 01:44:36 +07:00
0b1eaec3bc [tele5] Prefer jwplatform over nexx (closes #25533) 2020-06-06 01:35:09 +07:00
b37e47a3f9 [twitch:stream] Expect 400 and 410 HTTP errors from API 2020-06-06 00:57:40 +07:00
ce3735df02 [twitch:stream] Fix extraction (closes #25528) 2020-06-06 00:55:29 +07:00
a0455d0ffd [twitch] Pass v5 accept header and fix thumbnails extraction (closes #25531) 2020-06-06 00:12:47 +07:00
c8b232cc48 [brightcove] Sort imports 2020-06-05 23:35:57 +07:00
b4eb0bc7bd [brightcove] Fix subtitles extraction (closes #25540) 2020-06-05 23:33:14 +07:00
d5147b65ac [malltv] Add support for sk.mall.tv (#25445) 2020-06-01 21:11:31 +07:00
7b0b53ea69 [twitter:broadcast] Add untitled periscope broadcast test 2020-06-01 20:32:57 +07:00
7016e24ebe [periscope] Fix untitled broadcasts (#25482) 2020-06-01 20:31:51 +07:00
bef4688c72 [jwplatform] Improve embeds extraction (closes #25467) 2020-05-31 11:10:31 +07:00
228c1d685b release 2020.05.29 2020-05-29 03:33:13 +07:00
efd72b05d2 [ChangeLog] Actualize
[ci skip]
2020-05-29 03:28:44 +07:00
fe515e5c75 [ard:beta] Extend _VALID_URL (closes #25405) 2020-05-29 02:01:51 +07:00
1db5ab6b34 [youtube] Add support for more invidious instances (#25417) 2020-05-27 01:26:45 +07:00
2791e80b60 [postprocessor/ffmpeg] Embed series metadata with --add-metadata 2020-05-23 12:28:15 +07:00
8f841fafcd [giantbomb] Extend _VALID_URL (#25222) 2020-05-21 04:30:50 +07:00
a54c5f83c0 [ard] Improve _VALID_URL (closes #25134) (#25198) 2020-05-20 04:08:08 +07:00
cd13343ad8 [redtube] Improve formats extraction and extract m3u8 formats (closes #25311, closes #25321) 2020-05-20 03:39:41 +07:00
Rob
9cd5f54e31 [utils] Fix file permissions in write_json_file (closes #12471) (#25122) 2020-05-20 03:21:52 +07:00
9a269547f2 [indavideo] Switch to HTTPS for API request (#25191) 2020-05-20 02:13:06 +07:00
bf097a5077 [redtube] Improve title extraction (#25208) 2020-05-20 02:11:05 +07:00
52c50a10af [vimeo] improve format extraction and sorting(closes #25285) 2020-05-15 15:57:06 +01:00
b334732709 [soundcloud] reduce API playlist page limit(closes #25274) 2020-05-15 14:13:02 +01:00
384bf91f88 [youtube] Add support for yewtu.be (#25226) 2020-05-14 05:54:42 +07:00
fae11394f0 [README.md] flake8 HTTPS URL (#25230) 2020-05-14 05:53:17 +07:00
adc13b0748 [mailru] Fix extraction (closes #24530) (#25239) 2020-05-14 05:51:40 +07:00
327593257c [bbccouk] PEP8 2020-05-14 05:11:42 +07:00
9d8f3a12a6 [spike] fix Bellator mgid extraction(closes #25195) 2020-05-12 20:49:08 +01:00
b002bc433a release 2020.05.08 2020-05-08 18:10:37 +07:00
b74896dad1 [ChangeLog] Actualize
[ci skip]
2020-05-08 18:07:05 +07:00
fa3db38333 [youtube] Improve signature cipher extraction (closes #25188) 2020-05-08 17:42:30 +07:00
30fa5c6087 [iprima] Improve extraction (closes #25138) 2020-05-06 23:20:14 +07:00
6c907eb33f [downloader/http] Request last data block of exact remaining size
Always request last data block of exact size remaining to download if possible not the current block size.
2020-05-05 21:43:39 +07:00
f7b42518dc [downloader/http] Finish downloading once received data length matches expected
Always do this if possible, i.e. if Content-Length or expected length is known, not only in test.
This will save unnecessary last extra loop trying to read 0 bytes.
2020-05-05 21:43:39 +07:00
ce7db64bf1 [uol] fix extraction(closes #22007) 2020-05-05 11:19:40 +01:00
1328305851 [orf] Add support for more radio stations (closes #24938) (#24968) 2020-05-05 06:22:50 +07:00
6c22cee673 [extractor/common] Use compat_cookiejar_Cookie for _set_cookie (closes #23256, closes #24776)
To always ensure cookie name and value are bytestrings on python 2.
2020-05-05 06:00:37 +07:00
6d874fee2a [compat] Introduce compat_cookiejar_Cookie 2020-05-05 05:54:10 +07:00
676723e0da [dailymotion] Fix typo 2020-05-05 05:09:07 +07:00
c380cc28c4 [utils] Improve cookie files support
+ Add support for UTF-8 in cookie files
* Skip malformed cookie file entries instead of crashing (invalid entry len, invalid expires at)
2020-05-05 04:21:25 +07:00
f7f304910d [puhutv] Remove no longer available HTTP formats (closes #25124) 2020-05-04 21:15:19 +07:00
00a41ca4c3 release 2020.05.03 2020-05-03 00:05:05 +07:00
66f32ca0e1 [ChangeLog] Actualize
[ci skip]
2020-05-02 23:59:25 +07:00
6ffc3cf74a [crunchyroll] Fix and improve extraction (closes #25096, closes #25060) 2020-05-02 23:42:51 +07:00
4433bb0245 [extractor/common] Extract multiple JSON-LD entries 2020-05-02 23:40:30 +07:00
e40c758c2a [youtube] Improve player id extraction and add tests 2020-05-02 07:18:08 +07:00
011e75e641 [youtube] Use redirected video id if any (closes #25063) 2020-05-01 00:40:38 +07:00
2468a6fa64 [yahoo] fix GYAO Player extraction and relax title URL regex(closes #24178)(closes #24778) 2020-04-29 14:56:32 +01:00
700265bfcf [tvplay] fix Viafree extraction(closes #15189)(closes #24473)(closes #24789) 2020-04-29 13:38:58 +01:00
c97f5e934f [tenplay] Relax _VALID_URL (closes #25001) 2020-04-26 12:41:33 +07:00
38db9a405a [prosiebensat1] Extract series metadata 2020-04-24 02:56:10 +07:00
2cdfe977d7 [prosiebensat1] Improve extraction and remove 7tv.de support (#24948) 2020-04-24 02:44:13 +07:00
46d0baf941 [options] Clarify doc on --exec command (closes #19087) (#24883) 2020-04-24 02:31:38 +07:00
00eb865b3c [youtube] Fix DRM videos detection (refs #24736) 2020-04-11 23:05:08 +07:00
2f19835726 [thisoldhouse] Improve video id extraction (closes #24549) 2020-04-11 20:07:37 +07:00
533f3e3557 [thisoldhouse] Fix video id extraction (closes #24548)
Added support for:
with of without "www."
and either  ".chorus.build" or ".com"

It now validated correctly on older URL's
```
<iframe src="https://thisoldhouse.chorus.build/videos/zype/5e33baec27d2e50001d5f52f
```
and newer ones
```
<iframe src="https://www.thisoldhouse.com/videos/zype/5e2b70e95216cc0001615120
```
2020-04-11 20:07:32 +07:00
75294a5ed0 [soundcloud] Improve AAC format extraction (closes #19173, closes #24708) 2020-04-10 17:26:03 +07:00
tom
b9e5f87291 [soundcloud] Extract AAC format 2020-04-10 17:25:04 +07:00
6b09401b0b [youtube] Skip broken multifeed videos (closes #24711) 2020-04-09 22:42:43 +07:00
5caf88ccb4 [nova:embed] Fix extraction (closes #24700) 2020-04-09 03:52:29 +07:00
dcc8522fdb [motherless] Fix extraction (closes #24699) 2020-04-09 02:14:49 +07:00
c9595ee780 [twitch:clips] Extend _VALID_URL (closes #24290) (#24642) 2020-04-07 23:21:25 +07:00
91bd3bd019 [tv4] Fix ISM formats extraction (closes #24667) 2020-04-07 22:56:06 +07:00
13b08034b5 [extractor/common] Skip malformed ISM manifest XMLs while extracting ISM formats (#24667) 2020-04-07 22:55:59 +07:00
6a6e1a0cd8 [tele5] Fix extraction (closes #24553) 2020-04-06 02:05:06 +07:00
4e7b5bba5f [mofosex] Add support for generic embeds (closes #24633) 2020-04-06 01:29:58 +07:00
52c4c51556 [youporn] Add support form generic embeds 2020-04-05 20:56:14 +07:00
8fae1a04eb [spankwire] Add support for generic embeds (refs #24633) 2020-04-05 20:42:56 +07:00
d44a707fdd [spankwire] Fix extraction (closes #18924, closes #20648) 2020-04-05 20:42:56 +07:00
049c0486bb release 2020.03.24 2020-03-24 03:14:30 +07:00
30b5121a1c [ChangeLog] Actualize
[ci skip]
2020-03-24 03:12:15 +07:00
b439634f0e [ChangeLog] Actualize
[ci skip]
2020-03-24 03:07:34 +07:00
6e47200b6e [teachable] Update test 2020-03-24 02:57:53 +07:00
38fa761a45 [teachable] Update gns3 domain 2020-03-24 02:57:48 +07:00
08a27407c4 [teachable] Update upskillcourses domain
New version does not use teachable platform any longer
2020-03-24 02:57:44 +07:00
be7dacf9cf [generic] Look for teachable embeds before wistia 2020-03-24 02:57:38 +07:00
4560adc820 [teachable] Extract chapter metadata (closes #24421) 2020-03-24 02:57:32 +07:00
63dce3094b [bilibili] Add support for player.bilibili.com (closes #24402) 2020-03-24 00:24:39 +07:00
b4eb08bb03 [bilibili] Add support for new URL schema with BV ids (closes #24439, closes #24442) 2020-03-24 00:11:39 +07:00
2e20cb3636 [limelight] remove disabled API requests(closes #24255) 2020-03-23 12:57:10 +01:00
a6c5859d6b [soundcloud] fix download url extraction(closes #24394) 2020-03-22 09:24:26 +01:00
c76cdf2382 [cbc:watch] Fix authenticated device token caching (closes #19160) 2020-03-21 01:43:13 +07:00
787c360467 [cbc:watch] Add support for authentication 2020-03-21 01:43:08 +07:00
73453430c1 [hellporno] Fix extraction (closes #24399) 2020-03-21 00:59:48 +07:00
158bc5ac03 [xtube] Fix typo 2020-03-14 22:58:10 +07:00
4568a11802 [xtube] Fix formats extraction (closes #24348) 2020-03-14 22:57:10 +07:00
4cbce88f8b [ndr] Fix extraction (closes #24326) 2020-03-14 04:58:24 +07:00
541fe3eaff [nhk] Update m3u8 URL and use native hls (#24329) 2020-03-14 04:42:40 +07:00
9bfe088594 [nhk] Remove obsolete rtmp formats (closes #24329) 2020-03-14 04:40:11 +07:00
fcaf4d7a06 [nhk] Relax _VALID_URL (#24329) 2020-03-14 04:39:21 +07:00
40b6495d40 Revert "[vimeo] fix showcase password protected video extraction(closes #24224)"
This reverts commit 12ee431676.
2020-03-13 08:59:10 +01:00
f1a8511f7b [utils] Add reference to cookie file format 2020-03-10 04:59:02 +07:00
042b664933 Revert "[utils] Add support for cookies with spaces used instead of tabs"
According to [1] TABs must be used as separators between fields.
Files produces by some tools with spaces as separators are considered
malformed.

1. https://curl.haxx.se/docs/http-cookies.html

This reverts commit cff99c91d1.
2020-03-10 04:53:51 +07:00
68fa15155f release 2020.03.08 2020-03-08 18:27:20 +07:00
434f573046 [ChangeLog] Actualize
[ci skip]
2020-03-08 18:16:17 +07:00
cff99c91d1 [utils] Add support for cookies with spaces used instead of tabs 2020-03-08 18:01:32 +07:00
fa9b8c6628 [pornhub] Add support for pornhubpremium.com (#24288) 2020-03-08 18:00:25 +07:00
ea782aca52 [README.md] Clarify 429 error 2020-03-08 09:17:17 +07:00
43ebf77df3 [youtube] Remove outdated code
Additional get_video_info requests don't seem to provide any extra itags any longer
2020-03-08 08:59:58 +07:00
d332ec725d [youtube] Improve age-gated videos extraction in 429 error conditions (refs #24283) 2020-03-08 05:41:04 +07:00
f93abcf1da [youtube] Improve extraction in 429 error conditions (closes #24283) 2020-03-08 05:09:02 +07:00
0ec9d4e565 [nhk] update API version(closes #24270) 2020-03-06 20:13:28 +01:00
34525a3885 release 2020.03.06 2020-03-06 00:25:43 +07:00
2db9ac228d [ChangeLog] Actualize
[ci skip]
2020-03-06 00:23:14 +07:00
5429d6a9cb [youtube] Fix tests 2020-03-06 00:05:50 +07:00
dc879c5a37 [youtube] Fix age-gated videos support without login (closes #24248) 2020-03-05 23:48:25 +07:00
12ee431676 [vimeo] fix showcase password protected video extraction(closes #24224) 2020-03-03 12:33:57 +01:00
46cc54ca8f [pornhub] Improve title extraction (closes #24184) 2020-03-03 06:23:39 +07:00
1e1c1960aa [peertube] Fix issues and improve extraction (closes #23657) 2020-03-03 03:01:47 +07:00
ac379fa236 [peertube] Improve extraction 2020-03-03 03:01:42 +07:00
jxu
0e30a7b973 [youtube:playlist] Fix tests (closes #23872) (#23885) 2020-03-03 01:46:00 +07:00
3b5399ce0f [servus] Add support for new URL schema (closes #23475, closes #23583, closes #24142) 2020-03-03 01:41:53 +07:00
1c45ff5572 [vimeo] Fix subtitles URLs (#24209) 2020-03-03 01:27:40 +07:00
669625a32c release 2020.03.01 2020-03-01 20:11:32 +07:00
170f5b7c27 [ChangeLog] Actualize
[ci skip]
2020-03-01 20:09:05 +07:00
b274e48d56 [xhamster] Fix extraction (closes #24205) 2020-03-01 20:04:48 +07:00
50d19895a1 [franceculture] Fix extraction (closes #24204) 2020-03-01 19:22:09 +07:00
6d475d01d8 [telecinco] Add support for article opening videos 2020-03-01 03:09:19 +07:00
f8cbd8c963 [telecinco] Fix extraction (refs #24195) 2020-03-01 01:04:51 +07:00
838f051c4b [xtube:user] Fix test 2020-02-29 23:51:56 +07:00
e88b450771 [xtube] Fix metadata extraction (closes #21073, closes #22455) 2020-02-29 23:51:34 +07:00
278355bae4 [zapiks] Fix test 2020-02-29 23:09:13 +07:00
b4cbdbd4b3 [zdf:channel] Fix tests 2020-02-29 23:06:36 +07:00
ea17979d83 [test_subtitles] Remove obsolete test 2020-02-29 22:08:43 +07:00
886d985959 [youjizz] Fix extraction (closes #24181) 2020-02-29 21:58:22 +07:00
7947a1f7db Remove no longer needed compat_str around geturl 2020-02-29 19:19:24 +07:00
fca6dba8b8 [YoutubeDL] Force redirect URL to unicode on python 2 2020-02-29 19:08:44 +07:00
e2f8bf5888 [extractor/common] Convert ISM manifest to unicode before processing on python 2 (#24152) 2020-02-29 17:29:30 +07:00
b76f0e58f7 [options] Remove duplicate short option -v for --version (#24162) 2020-02-29 16:33:09 +07:00
bee6451fe8 [pornhd] Fix extraction (closes #24128) 2020-02-24 04:47:56 +07:00
00d798b7c2 [teachable] Add support for multiple videos per lecture (closes #24101) 2020-02-23 06:49:45 +07:00
fda6d237a5 [wistia] Add support for multiple generic embeds (closes #8347, closes #11385) 2020-02-23 06:47:11 +07:00
5d9f6cbc5a [imdb] Fix extraction (closes #23443) 2020-02-23 04:33:29 +07:00
97c822b3d5 [tv2dk:bornholm:play] Fix extraction (#24076) 2020-02-19 01:02:05 +07:00
117ba9e9df release 2020.02.16 2020-02-16 22:43:42 +07:00
0d718db623 [ChangeLog] Actualize
[ci skip]
2020-02-16 22:40:44 +07:00
7bf27721d6 [npr] Add support for streams (closes #24042) 2020-02-15 05:35:55 +07:00
f6052ec923 [24video] Add support for porn.24video.net (closes #23779, closes #23784) 2020-02-15 03:49:29 +07:00
4e9e1e240d [test_YoutubeDL] Add tests for #10591 (closes #23873) 2020-02-15 03:37:31 +07:00
e0abaab293 [test_YoutubeDL] Fix get_ids 2020-02-15 03:37:25 +07:00
jxu
de1121d749 [YoutubeDL] Fix playlist entry indexing with --playlist-items (closes #10591, closes #10622) 2020-02-15 03:36:53 +07:00
293c9f0186 [jpopsuki] Remove extractor (closes #23858) 2020-02-15 02:23:29 +07:00
06f1de2daf [nova] Improve extraction (refs #23690) 2020-02-15 02:16:26 +07:00
b68a6e32fb [nova:embed] Improve (closes #23690) 2020-02-15 02:00:58 +07:00
8cd809fb3d [nova:embed] Fix extraction (closes #23672) 2020-02-15 02:00:52 +07:00
d6aa1db7ed [abc:iview] Support 720p (#22907) (#22921) 2020-02-13 14:52:00 +01:00
f377edec06 [nytimes] improve format sorting(closes #24010) 2020-02-10 09:43:20 +01:00
bfe2b8cf2a [update] Fix updating via symlinks (closes #23991) 2020-02-08 19:46:58 +07:00
82fea5b42e [compat] Introduce compat_realpath (refs #23991) 2020-02-08 19:36:55 +07:00
fffc618c51 [toggle] Add support for mewatch.sg (closes #23895) (#23930) 2020-02-05 22:41:56 +07:00
705b1cda99 [thisoldhouse] fix extraction(closes #23951) 2020-02-03 13:20:36 +01:00
7d55b62ff2 [popcorntimes] Add extractor (closes #23949) 2020-02-03 06:05:56 +07:00
0d006fac5c [sportdeutschland] Update to new sportdeutschland API
They switched to SSL, but under a different host AND path...
Remove the old test cases because these videos have become unavailable.
2020-02-01 23:35:55 +01:00
00de61a98f [twitch:stream] Lowercase channel id for stream request (closes #23917) 2020-02-01 00:32:25 +07:00
d95a1cc98e [tv5mondeplus] Fix extraction (closes #23907, closes #23911) 2020-01-31 04:58:36 +07:00
4935749730 [tva] Relax _VALID_URL (closes #23903) 2020-01-31 03:49:16 +07:00
51c7f40c83 [vimeo] fix album extraction(closes #23864) 2020-01-27 23:37:29 +01:00
4877ffc0e9 [viewlift] improve extraction
- fix extraction(closes #23851)
- add add support for authentication
- add support for more domains
2020-01-27 15:41:21 +01:00
8e4d3f83ce [svt] fix series extraction(closes #22297) 2020-01-26 16:17:51 +01:00
43e7994749 [svt] fix article extraction(closes #22897)(closes #22919) 2020-01-26 14:16:59 +01:00
2a5c26c980 [soundcloud] imporve private playlist/set tracks extraction
https://github.com/ytdl-org/youtube-dl/issues/3707#issuecomment-577873539
2020-01-23 23:24:37 +01:00
76dbe4df5f release 2020.01.24 2020-01-24 04:16:05 +07:00
bffdedfabd [ChangeLog] Actualize
[ci skip]
2020-01-24 04:14:08 +07:00
c3cfea9068 [youtube] Fix sigfunc name extraction (closes #23819) 2020-01-24 04:09:10 +07:00
22cb94902f [stretchinternet] fix extraction(closes #4319) 2020-01-19 21:20:56 +01:00
be96f9924f [voicerepublic] fix extraction 2020-01-19 20:15:02 +01:00
9cf30dc017 [azmedien] fix extraction(closes #23783) 2020-01-19 19:30:48 +01:00
f4a18db748 [ard] add a missing condition 2020-01-19 18:28:24 +01:00
PB
fd032450f0 [businessinsider] Fix jwplatform id extraction (closes #22929) (#22954) 2020-01-18 22:47:50 +07:00
a4b2769451 [24video] Add support for 24video.vip (closes #23753) 2020-01-18 15:05:45 +07:00
d9a2f86791 [ivi:compilation] Fix entries extraction (closes #23770) 2020-01-18 14:46:38 +07:00
c968f738df [ard] improve extraction(closes #23761)
- simplify extraction
- extract age limit and series
- bypass geo-restriction
2020-01-17 14:23:24 +01:00
48ff5590c1 [nbc] add support for nbc multi network URLs(closes #23049) 2020-01-16 15:37:16 +01:00
2c482bff7c [americastestkitchen] fix extraction 2020-01-15 14:18:04 +01:00
a9866c0366 [zype] improve extraction
- extract subtitles(closes #21258)
- support URLs with alternative keys/tokens(#21258)
- extract more metadata
2020-01-15 14:18:04 +01:00
90ea83c64d [orf:tvthek] Improve geo restricted videos detection (closes #23741) 2020-01-15 04:32:05 +07:00
e4e5fa6e3c [soundcloud] Restore previews extraction (closes #23739) 2020-01-15 04:13:10 +07:00
e8cf0dbdd8 release 2020.01.15 2020-01-15 01:37:29 +07:00
d7c55f226d [ChangeLog] Actualize
[ci skip]
2020-01-15 01:34:01 +07:00
bfdc8340c9 [yourporn] Fix extraction (closes #21645, closes #22255, closes #23459) 2020-01-15 01:28:17 +07:00
14bb191634 [travis] Add flake8 job (#23720) 2020-01-15 01:09:08 +07:00
628e5bc0b7 [canvas] Add support for new API endpoint and update tests (closes #17680, closes #18629) 2020-01-14 23:53:59 +07:00
3fc56635b7 [ndr:base:embed] Improve thumbnails extraction (closes #23731) 2020-01-14 21:46:56 +07:00
bd2c211fcc [vodplatform] add support for embed.kwikmotion.com domain 2020-01-12 17:34:57 +01:00
10a5091e58 [twitter] add support for promo_video_website cards(closes #23711) 2020-01-12 12:01:59 +01:00
aca2fd222f [orf:radio] Clean description and improve extraction 2020-01-11 02:18:36 +07:00
9ba179c1fa [orf:fm4] Fix extraction (#23599) 2020-01-11 01:51:15 +07:00
3fdf573148 [safari] Fix kaltura session extraction (closes #23679) (#23670) 2020-01-11 01:34:26 +07:00
d4e0cd69ef [lego] fix extraction and extract subtitle(closes #23687) 2020-01-10 05:06:45 +01:00
483b858d49 [cloudflarestream] import embed URL extraction 2020-01-08 23:07:41 +01:00
a71c1d1a5a [cloudflarestream] improve extraction
- add support for bytehighway.net domain
- add support for signed URLs
- extract thumbnail
2020-01-08 22:42:53 +01:00
838171630d [naver] improve metadata extraction 2020-01-08 12:55:33 +01:00
c88debff5d [naver] improve extraction
- improve geo-restriction handling
- extract automatic captions
- extract uploader metadata
- extract VLive HLS formats
2020-01-08 10:59:56 +01:00
3cb05b86de [pandatv] Remove extractor (#23630) 2020-01-07 21:11:03 +07:00
b2771a2853 [dctp] fix format extraction(closes #23656) 2020-01-07 13:03:32 +01:00
7bac77413d [scrippsnetworks] correct test case URL 2020-01-06 14:30:02 +01:00
0264903574 [scrippsnetworks] add support for www.discovery.com videos 2020-01-06 14:25:54 +01:00
2f7aa680b7 [discovery] fix anonymous token extraction(closes #23650) 2020-01-06 14:25:54 +01:00
0d2306d02b [nrktv:seriebase] Fix extraction (closes #23625) (#23537) 2020-01-06 06:34:36 +07:00
233826f68f [wistia] improve format extraction and extract subtitles(closes #22590) 2020-01-05 21:09:37 +01:00
259ad38173 [devscripts/create-github-release] Remove unused import 2020-01-06 01:26:22 +07:00
44b434e4e3 [vice] improve extraction(closes #23631) 2020-01-05 16:33:21 +01:00
484637a9cc [redtube] Detect private videos (#23518) 2020-01-02 22:45:42 +07:00
ca069f6881 release 2020.01.01 2020-01-01 05:24:58 +07:00
0d5c415e1f [devscripts/create-github-release] Switch to using PAT for authentication
Basic authentication will be deprecated soon
2020-01-01 05:20:48 +07:00
d6bf9cbd46 [ChangeLog] Actualize
[ci skip]
2020-01-01 04:13:32 +07:00
de7aade2f8 [soundcloud] fix client id extraction for non fatal requests 2019-12-31 21:31:22 +01:00
2d30b92e11 [brightcove] invalidate policy key cache on failing requests 2019-12-31 19:49:01 +01:00
0164cd5dac [pornhub] Improve locked videos detection (closes #22449, closes #22780) 2019-12-31 23:43:43 +07:00
f41347260c [pornhub] Fix extraction and add support for m3u8 formats (closes #22749, closes #23082) 2019-12-31 23:29:06 +07:00
0606808746 [brightcove] update policy key on failing requests 2019-12-31 16:44:30 +01:00
0a02732b56 [spankbang] Improve removed video detection (#23423) 2019-12-31 22:18:01 +07:00
2b845c4086 [spankbang] Fix extraction (closes #23307, closes #23423, closes #23444) 2019-12-31 22:16:39 +07:00
3bed621750 [soundcloud] automatically update client id on failing requests 2019-12-31 09:49:29 +01:00
0c15a56f1c [prosiebensat1] improve geo restriction handling(closes #23571) 2019-12-30 22:31:11 +01:00
75ef77c1b1 [brightcove] cache brightcove player policy keys 2019-12-29 19:31:17 +01:00
cb7e053e0a [extractors] add missing import for ScrippsNetworksIE 2019-12-29 19:31:17 +01:00
941e359e95 [teachable] Fail with error message if no video URL found 2019-12-27 00:26:12 +07:00
f8a12427a9 [teachable] Improve locked lessons detection (#23528) 2019-12-27 00:18:37 +07:00
7ea55819ac [scrippsnetworks] Add new extractor(closes #19857)(closes #22981) 2019-12-26 15:25:04 +01:00
18ff573e50 [mitele] fix extraction(closes #21354)(closes #23456) 2019-12-25 20:02:31 +01:00
d1b2722095 [soundcloud] Update client id (closes #23516) 2019-12-25 22:39:50 +07:00
278be57be2 [mailru] Relax _VALID_URLs (#23509) 2019-12-25 04:28:34 +07:00
80e43af5bf release 2019.12.25 2019-12-25 01:16:49 +07:00
b1a92520a3 [ChangeLog] Actualize
[ci skip]
2019-12-25 00:52:11 +07:00
9b6e72fd06 [mediaset] Fix parse formats (closes #23508) 2019-12-24 23:51:08 +07:00
2dbc0967f2 [ChangeLog] Actualize
[ci skip]
2019-12-16 00:40:34 +07:00
fab01080f4 [tv2dk:bornholm:play] Add extractor (closes #23291) 2019-12-16 00:08:18 +07:00
42db58ec73 [utils] Improve str_to_int 2019-12-15 23:15:24 +07:00
73d8f3a634 [slideslive] add support for url and vimeo service names(closes #23414) 2019-12-14 21:35:31 +01:00
b33a05d221 [slideslive] fix extraction(closes #23413) 2019-12-14 19:29:04 +01:00
232ed8e6e0 [twitch] fix clip extraction(closes #23375) 2019-12-13 11:00:31 +01:00
cf80ff186e [soundcloud] add support for token protected embeds(#18954) 2019-12-09 14:38:12 +01:00
0e6ec3caf6 [vk] improve extraction
- fix User Videos extraction(closes #23356)
- extract all videos for lists with more than 1000 videos(#23356)
- add support for video albums(closes #14327)(closes #14492)
2019-12-09 09:13:02 +01:00
d686cab084 [kontrtube] remove extractor 2019-12-08 12:38:21 +01:00
9d4424afaa [videopremium] remove extractor 2019-12-08 11:54:16 +01:00
ce709fcb00 [musicplayon] remove extractor(closes #9225) 2019-12-07 20:17:30 +01:00
6633103f8e [ufctv] add support for ufcfightpass.imgdge.com and ufcfightpass.imggaming.com domains(closes #23343) 2019-12-07 19:23:19 +01:00
1d31b7ca04 [twitch] extract m3u8 formats frame rate(closes #23333) 2019-12-06 15:34:35 +01:00
4067a23270 [ufctv] add support for more domains and remove compatibility code(closes #23332) 2019-12-06 11:04:12 +01:00
7d53fa475a [imggaming] add support for playlists and extract subtitles 2019-12-04 20:56:23 +01:00
3ae878605d [ufctv] fix extraction and add support for UFC Arabia(closes #23312) 2019-12-04 17:20:53 +01:00
22974a3782 [yahoo] correct gyao brightcove player id(closes #23303) 2019-12-03 21:13:44 +01:00
63fe44eb4d [vzaar] update test 2019-12-03 12:31:16 +01:00
c712b16dc4 [vzaar] override AES decryption key URL(closes #17521) 2019-12-03 12:23:08 +01:00
6797de75e0 [vzaar] add support for AES HLS manifests(closes #17521)(closes #23299) 2019-12-03 11:37:30 +01:00
12cc89122d [nrl] fix extraction 2019-11-30 23:50:28 +01:00
3765284476 [teachingchannel] fix extraction 2019-11-30 23:49:45 +01:00
ddfe50195b [nintendo] fix extraction and partially add support for Nintendo Direct videos(#4592) 2019-11-30 23:48:26 +01:00
1ed2c4b378 [ooyala] add better fallback values for domain and streams variables 2019-11-30 23:21:13 +01:00
66b4872747 [youtube] add support youtubekids.com(closes #23272) 2019-11-30 17:51:34 +01:00
0b25af9bf5 [tv2] detect DRM protection 2019-11-30 15:50:17 +01:00
8d3a3a9901 [tv2] add support for mtv.fi and fix tv2.no article extraction(closes #10543) 2019-11-30 15:26:12 +01:00
c0b1e01330 [msn] improve extraction
- add support for YouTube and NBCSports embeds
- add support for aricles with multiple videos
- improve AOL embed support
- improve format extraction
2019-11-29 17:39:18 +01:00
88a7a9089a [abcotvs] relax _VALID_URL regex and improve metadata extraction(closes #18014) 2019-11-29 17:39:18 +01:00
a15adbe461 [channel9] reduce response size and update tests 2019-11-29 17:39:18 +01:00
7f641d2c7a [adobetv] improve extaction
- use OnDemandPagedList for list extractors
- reduce show extraction requests
- extract original video format and subtitles
- add support for adobe tv embeds
2019-11-29 17:39:18 +01:00
348c6bf1c1 [utils] handle int values passed to str_to_int 2019-11-29 17:39:18 +01:00
b568561eba release 2019.11.28 2019-11-28 23:25:25 +07:00
e3f00f139f [ChangeLog] Actualize
[ci skip]
2019-11-28 23:09:48 +07:00
681ac7c92a [vimeo] improve extraction
- fix review extraction
- fix ondemand extraction
- make password protected player case as an expected error(closes #22896)
- simplify channel based extractors code
2019-11-27 13:57:30 +01:00
6471d0d3b8 [openload] remove OpenLoad related extractors(closes #11999)(closes #15406) 2019-11-26 23:57:37 +01:00
5ef62fc4ce [dailymotion] improve extraction
- extract http formats included in m3u8 manifest
- fix user extraction(closes #3553)(closes #21415)
- add suport for User Authentication(closes #11491)
- fix password protected videos extraction(closes #23176)
- respect age limit option and family filter cookie value(closes #18437)
- handle video url playlist query param
- report alowed countries for geo-restricted videos
2019-11-26 22:18:21 +01:00
df65a4a1ed [corus] improve extraction
- add support for Series Plus, W Network, YTV, ABC Spark, disneychannel.com
  and disneylachaine.ca(closes #20861)
- add support for self hosted videos(closes #22075)
- detect DRM protection(closes #14910)(closes #9164)
2019-11-26 22:18:21 +01:00
edc2a1f68b [vivo] Fix extraction (closes #22328, closes #22279) 2019-11-27 02:28:06 +07:00
1ced222120 [utils] Add generic caesar cipher and rot47 2019-11-27 02:26:42 +07:00
6ddd4bf6ac [bitchute] Extract upload date (closes #22990) (#23193) 2019-11-27 00:20:39 +07:00
9d30c2132a [utils] Handle rd-suffixed day parts in unified_strdate (#23199) 2019-11-27 00:08:37 +07:00
cf3c9eafad [soundcloud] Update client id (closes #23214) 2019-11-27 00:03:51 +07:00
0de9fd24dc release 2019.11.22 2019-11-22 01:24:27 +07:00
fb8dfc5a27 [ChangeLog] Actualize
[ci skip]
2019-11-22 01:21:00 +07:00
80a51fc2ef [ivi] Skip s353 for bundled exe
See https://github.com/Legrandin/pycryptodome/issues/228
2019-11-22 01:10:24 +07:00
f8015c1574 [ivi] Fix python 3.4 support 2019-11-21 23:38:39 +07:00
25d3f770e6 [ivi] Ask for pycryptodomex instead of pycryptodome
See discussion at 1bba88efc7 (r35982110)
2019-11-21 23:22:59 +07:00
f0f6a7e73f [chaturbate] Fix extraction (closes #23010, closes #23012) 2019-11-21 23:21:03 +07:00
76d9eca43d [ivi] fallback to old extraction method for unknown error codes 2019-11-19 20:16:31 +01:00
f9c4a45210 [ntvru] add support for non relative file URLs(closes #23140) 2019-11-18 21:40:53 +01:00
7e70620a34 [vk] fix wall audio thumbnails extraction(closes #23135) 2019-11-18 12:51:25 +01:00
9e4e864639 [ivi] improve error detection 2019-11-16 01:51:48 +01:00
6c79785bb0 [travis] Add python 3.8 build 2019-11-16 07:47:23 +07:00
7360c06fac [extractor/common] Add data, headers and query to all major extract methods preserving standard order for potential future use 2019-11-16 05:55:54 +07:00
1bba88efc7 [ivi] sign content request only when pycryptodome is available 2019-11-15 23:46:31 +01:00
656c20010f [ivi] fix format extraction(closes #21991) 2019-11-15 21:17:47 +01:00
8b1a30c993 [comcarcoff] remove extractor 2019-11-14 06:39:21 +01:00
5709d661a2 [drtv] Add support for new URL schema (closes #23059) 2019-11-14 01:45:04 +07:00
eb22d1b557 [nexx] Add support for Multi Player JS Setup(closes #23052) 2019-11-13 19:09:32 +01:00
48970d5cc8 [teamcoco] add support for new videos(closes #23054) 2019-11-12 10:51:54 +01:00
2e9ad59a4d [soundcloud] check if the soundtrack has downloads left(closes #23045) 2019-11-11 09:53:04 +01:00
433e071058 [facebook] fix posts video data extraction(closes #22473) 2019-11-10 17:02:47 +01:00
9e46d1f8aa [addanime] remove extractor 2019-11-09 17:15:15 +01:00
88b87b08b1 [minhateca] remove extractor 2019-11-09 17:01:21 +01:00
20baa17c01 [daisuki] remove extractor 2019-11-09 16:00:12 +01:00
8fbf5d2f87 [seeker] remove Revision3 extractors and fix extraction 2019-11-09 13:14:23 +01:00
f81dd65ba2 [extractor/common] clean jwplayer description HTML tags 2019-11-09 13:11:59 +01:00
ce112a8c19 [twitch] fix video comments URL(#18593)(closes #15828) 2019-11-09 11:01:07 +01:00
18ca61c5e1 [twitter] improve extraction
- add support for generic embeds(closes #22168)
- always extract http formats for native videos(closes #14934)
- add support for Twitter Broadcasts(closes #21369)
- extract more metadata
- improve VMap format extraction
- unify extraction code for both twitter statuses and cards
2019-11-09 09:23:20 +01:00
0b16b3c2d3 [twitch] add support for Clip embed URLs 2019-11-09 09:22:24 +01:00
d4f53af482 [lnkgo] fix extraction(closes #16834) 2019-11-06 23:14:26 +01:00
5d92b407e0 [mixcloud] improve extraction
- improve metadata extraction(closes #11721)
- fix playlist extraction(closes #22378)
- fix user mixes extraction(closes #15197)(closes #17865)
2019-11-06 20:41:49 +01:00
55adb63e54 [kinja] add support for Kinja embeds
closes #5756
closes #11282
closes #22237
closes #22384
2019-11-06 19:56:10 +01:00
d64ec1242e [onionstudios] fix extraction 2019-11-06 10:44:19 +01:00
3ec86619e3 [common] initialize headers param with empty dict 2019-11-06 07:18:29 +01:00
57033e35e5 [common] fix typo 2019-11-05 23:41:57 +01:00
d7def23d05 [hotstar] pass Referer header to format requests(closes #22836) 2019-11-05 23:08:42 +01:00
b6139cb0c3 [common] pass headers to _extract_(m3u8|mpd)_formats methods 2019-11-05 22:56:25 +01:00
2318629b2b [dplay] minimize response size 2019-11-05 14:04:50 +01:00
b77c3949e8 [patreon] minimize reponse size and extract uploader_id and filesize 2019-11-05 14:04:17 +01:00
e9b95167af [roosterteeth] fix login request(closes #16094)(closes #22689) 2019-11-05 10:06:02 +01:00
ea07412ebf release 2019.11.05 2019-11-05 05:32:56 +07:00
1a4e4b0bfe [ChangeLog] Actualize
[ci skip]
2019-11-05 05:31:40 +07:00
20218040db [scte] Add extractor (closes #22975) 2019-11-05 05:21:16 +07:00
c69e71733d [msn] add support for Vidible and AOL embeds(closes #22195)(closes #22227) 2019-11-04 22:21:00 +01:00
3e49083604 [myspass] fix video URL extraction and improve metadata extraction(closes #22448) 2019-11-04 20:05:27 +01:00
2349255abd [jamendo] restore track url modification 2019-11-04 15:51:44 +01:00
e452345fc5 [jamendo] improve extraction
- fix album extraction(closes #18564)
- improve metadata extraction(closes #18565)(closes #21379)
2019-11-04 15:43:52 +01:00
bf45295c53 [mediaset] relax URL guid matching(closes #18352) 2019-11-04 11:13:14 +01:00
ef382405c6 [mediaset] extract unprotected M3U and MPD manifests(closes #17204) 2019-11-04 02:02:29 +01:00
a6e6673e82 [README.md] Also read permission to the binary in how to update section (#22903) 2019-11-04 04:23:27 +07:00
564275e26f [telegraaf] fix extraction 2019-11-03 22:04:03 +01:00
726e8eef59 [bellmedia] add support for marilyn.ca videos(#22193) 2019-11-02 22:33:51 +01:00
e54924c46f [stv] fix extraction(closes #22928) 2019-11-02 18:13:31 +01:00
5e36b63486 [iconosquare] remove extractor 2019-11-02 13:25:39 +01:00
9249c50c18 [keek] remove extractor 2019-11-02 13:09:44 +01:00
79b35e7c15 [gameone] Remove extractor (#21778) 2019-11-02 11:32:49 +00:00
836bfcb54e [flipagram] remove extractor 2019-11-02 11:08:51 +01:00
4c95fcf9e8 [bambuser] remove extractor
https://web.archive.org/web/20190808014227/https://go.bambuser.com/shutdown-announcement
2019-11-01 21:16:47 +01:00
152f22920d [wistia] reduce embed extraction false positives and support inline embeds(closes #22931) 2019-11-01 17:44:34 +01:00
20cc7c082b [go90] remove extractor 2019-11-01 16:36:35 +01:00
e987ce4bda [kakao] remove raw request and extract format total bitrate 2019-11-01 12:40:41 +01:00
d439989215 [daum] fix VOD and Clip extracton(closes #15015) 2019-11-01 11:43:18 +01:00
274bf5e4c5 [kakao] improve extraction
- support embed URLs
- support Kakao Legacy vid based embed URLs
- only extract fields used for extraction
- strip description and extract tags
2019-11-01 11:37:41 +01:00
e993f1a095 [mixcloud] fix cloudcast data extraction(closes #22821) 2019-10-31 08:13:10 +01:00
3cf70bf159 [yahoo] make cbs URL suffix part of the media alias 2019-10-31 07:44:21 +01:00
237513e801 [yahoo] restore support for cbs suffixed URLs 2019-10-31 07:38:53 +01:00
8040a0d35e [yahoo] fix typo 2019-10-30 23:52:09 +01:00
45f4a43389 [yahoo] improve extraction
- add support for live streams(closes #3597)(closes #3779)(closes #22178)
- bypass cookie consent page for european domains(closes #16948)(closes #22576)
- add generic support for embeds(closes #20332)
2019-10-30 23:24:49 +01:00
9a621ddc3a [tv2] Fix and improve extraction (closes #22787) 2019-10-30 02:21:52 +07:00
c56b2ac43c [tv2dk] Add extractor 2019-10-30 02:21:03 +07:00
8989349e6d [onet] improve extraction
- add support for onet100.vod.pl domain
- extract m3u8 formats
- correct audio only format info
2019-10-29 09:50:01 +01:00
7455832f31 [fox9] fix extraction 2019-10-29 09:50:00 +01:00
c4bd9cb7bb release 2019.10.29 2019-10-29 06:12:33 +07:00
cae0bbc538 [ChangeLog] Actualize
[ci skip]
2019-10-29 06:11:09 +07:00
53896ca5be [utils] Actualize major IPv4 address blocks per country 2019-10-29 06:10:20 +07:00
0d7392e68b [ChangeLog] Actualize
[ci skip]
2019-10-29 05:54:32 +07:00
aef9f87ea4 [go] Improve and beautify _VALID_URL 2019-10-29 05:52:15 +07:00
dd90a21c28 [go] Add support for abc.com and freeform.com (closes #22823, closes #22864) 2019-10-29 05:49:36 +07:00
01358b9fc1 [extractors] add import for MTVJapanIE 2019-10-28 23:34:31 +01:00
3cdcebf547 [mtv] add support for mtvjapan.com 2019-10-28 23:31:14 +01:00
cfabc50598 [mtv] fix extraction for mtv.de (closes #22113) 2019-10-28 22:55:01 +01:00
0086726e86 [videodetective] fix extraction 2019-10-28 19:48:34 +01:00
83e49259bf [internetvideoarchive] fix extraction 2019-10-28 19:47:27 +01:00
895e5c03db [nbcnews] fix extraction
closes #12569
closes #12576
closes #21703
closes #21923
2019-10-28 19:31:20 +01:00
702984eca9 [hark] remove extractor 2019-10-28 17:49:05 +01:00
b3c2fa6dad [tutv] remove extractor 2019-10-28 17:42:33 +01:00
831b732da1 [learnr] remove extractor 2019-10-28 17:41:17 +01:00
3e252cca0e [macgamestore] remove extractor
Covered by generic extractor
2019-10-28 17:39:01 +01:00
0f9d53566a [la7] update Kaltura service URL(closes #22358) 2019-10-28 15:17:06 +01:00
80c2126e80 [thesun] fix extraction(closes #16966) 2019-10-28 13:32:35 +01:00
71fa0b04f9 [makertv] remove extractor 2019-10-28 13:30:30 +01:00
dd90451f0f [tenplay] Add new extractor(closes #21446) 2019-10-27 22:02:46 +01:00
548c395716 [soundcloud] improve extraction
- improve format extraction(closes #22123)
- extract uploader_id and uploader_url(closes #21916)
- extract all known thumbnails(closes #19071)(closes #20659)
- fix extration for private playlists(closes #20976)
- add support for playlist embeds(#20976)
- skip preview formats(closes #22806)
2019-10-27 17:52:46 +01:00
0b98f3a751 [dplay] improve extraction
- add support for dplay.fi, dplay.jp and es.dplay.com(closes #16969)
- fix it.dplay.com extraction(closes #22826)
- update tests
- extract creator, tags and thumbnails
- handle playback API call errors
2019-10-26 14:58:29 +01:00
235dbb434b [discoverynetworks] add support for dplay.co.uk 2019-10-26 14:57:42 +01:00
42cd0824b3 [vk] remove assert statement 2019-10-26 00:06:05 +01:00
3c989818e7 [vk] improve extraction
- add support for Odnoklassniki embeds
- update tests
- extract more video from user lists(closes #4470)
- fix wall post audio extraction(closes #18332)
- improve error detection(closes #22568)
2019-10-25 19:35:07 +01:00
416c3ca7f5 [odnoklassniki] add support for Schemeless embed extraction 2019-10-25 19:27:28 +01:00
162bcc68dc [puhutv] improve extraction
- fix subtitles extraction
- transform HLS URLs to http URLs
- improve metadata extraction
2019-10-24 12:53:33 +01:00
07154c7930 [facebook] extract subtitles(closes #22777) 2019-10-22 17:59:14 +01:00
0c2d10d225 [globo] handle alternative hash signing method 2019-10-22 17:59:14 +01:00
820215f0e3 release 2019.10.22 2019-10-22 00:09:02 +07:00
b4818e3c7a [ChangeLog] Actualize
[ci skip]
2019-10-22 00:06:48 +07:00
2297c0d7d9 [facebook] Bypass download rate limits (closes #21018) 2019-10-19 23:56:36 +07:00
824fa51165 [utils] Improve subtitles_filename (closes #22753) 2019-10-18 04:03:53 +07:00
34e3885bc9 [viewster->contv] remove viewster extractor and add support for contv.com 2019-10-17 15:55:44 +01:00
59296bae7e [xfileshare] clean extractor
- update the list of domains
- add support for aa-encoded video data
- improve jwplayer format extraction
- add support for Clappr sources

closes #17032
closes #17906
closes #18237
closes #18239
2019-10-17 13:26:45 +01:00
755541a4c8 [mangomolo] fix video format extraction and add support for player URLs 2019-10-17 13:21:44 +01:00
86f63633c8 [audioboom] improve metadata extraction 2019-10-17 13:20:16 +01:00
0001157594 [atresplayer] Add coding cookie 2019-10-16 23:57:40 +01:00
bc48773ed4 [twitch] update VOD URL matching (closes #22395) (#22727) 2019-10-16 15:13:35 +00:00
d07866f13e [mit] Remove support for video.mit.edu(closes #22403) 2019-10-16 15:45:45 +01:00
2b115b9460 [servingsys] Remove extractor(closes #22639) 2019-10-16 15:41:58 +01:00
e29e96a9f5 [dumpert] fix extraction(closes #22428)(closes #22564) 2019-10-16 15:06:48 +01:00
6d394a66f5 [atresplayer] fix extraction(closes #16277)(closes #16716) 2019-10-16 12:04:52 +01:00
7815d6b743 release 2019.10.16 2019-10-16 03:26:47 +07:00
173190f5e3 [ChangeLog] Actualize
[ci skip]
2019-10-16 03:25:13 +07:00
974311b5aa [vimeo] improve album videos id extraction(closes #22599) 2019-10-15 21:01:59 +01:00
30eb05cb41 [globo] extract subtitles(closes #22713) 2019-10-15 19:54:53 +01:00
2af01c0293 [bokecc] improve player params extraction(closes #22638) 2019-10-15 15:18:51 +01:00
7e05df71b7 [nexx] handle result list(closes #22666) 2019-10-15 00:10:22 +01:00
a1ee23e98f [vimeo] fix VHX embed extraction 2019-10-14 18:37:35 +01:00
311ee45731 [nbc] switch to graphql api(closes #18581)(closes #22693)(closes #22701) 2019-10-14 18:36:25 +01:00
c317b6163b [vessel] remove extractor 2019-10-10 00:01:51 +01:00
2765c47a8c [promptfile] Remove extractor (closes #6239) 2019-10-10 03:40:01 +07:00
07b50f616e [kaltura] Fix service URL extraction (closes #22658) 2019-10-10 00:24:03 +07:00
1907f06e7b [kaltura] Fix embed info strip (refs #22658) 2019-10-10 00:11:41 +07:00
d4bb825b83 [globo] fix format extraction(closes #20319) 2019-10-09 11:08:28 +01:00
560d3b7d7c [redtube] Improve metadata extraction (closes #22492, closes #22615) 2019-10-05 22:04:49 +07:00
4bf568d36c [pornhub:uservideos:upload] Fix extraction (closes #22619) 2019-10-05 21:43:31 +07:00
05446d483d [telequebec:squat] Add support for squat.telequebec.tv (closes #18503) 2019-10-04 20:17:18 +07:00
3a37f2c3be [wimp] Remove extractor (closes #22088) (#22091) 2019-10-04 19:48:20 +07:00
0b87beefe6 [gfycat] Extend _VALID_URL (#22225) 2019-10-04 19:27:58 +07:00
fd4db1ebc2 [chaturbate] Extend _VALID_URL (#22309) 2019-10-04 19:22:01 +07:00
b64045cd2a [peertube] Update instances (#22414) 2019-10-04 19:17:16 +07:00
c2915de82e [telequebec] Add support for coucou.telequebec.tv (#22482) 2019-10-04 19:14:31 +07:00
4e72d02f39 [xvideos] Extend _VALID_URL (#22471) 2019-10-04 19:05:35 +07:00
76e510b92c [youtube] Remove support for invidious.enkirton.net (#22543) 2019-10-04 19:01:03 +07:00
9679a62a28 [openload] Add support for oload.monster (#22592) 2019-10-04 18:57:51 +07:00
ca20b13048 [nrktv:seriebase] Fix extraction (#22596) 2019-10-04 18:57:18 +07:00
894b3826f5 [youtube] Add support for yt.lelux.fi (#22597) 2019-10-04 18:52:15 +07:00
aaf9d904aa [orf:tvthek] Make manifest requests non fatal (refs #22578) 2019-10-03 00:55:46 +07:00
25e911a968 [extractor/common] Make _is_valid_url more relaxed 2019-10-03 00:53:07 +07:00
74bc299453 [teachable] Skip login when already logged in (closes #22572) 2019-10-02 02:03:22 +07:00
2906631e12 [viewlift] Fix URL matching 2019-10-01 23:18:11 +07:00
326ae4ff96 [viewlift] Improve extraction (closes #22545) 2019-09-29 23:03:39 +07:00
72fd4d0c6a [nonktube] Fix extraction (closes #22544) 2019-09-29 21:57:08 +07:00
f4b865c613 release 2019.09.28 2019-09-28 00:30:30 +07:00
412f44f4b3 [ChangeLog] Actualize
[ci skip]
2019-09-28 00:23:25 +07:00
6483fbd336 [vk] Fix extraction (closes #22522) 2019-09-28 00:04:52 +07:00
8130ac42e5 [openload] PEP 8 2019-09-26 23:15:06 +07:00
cb3e4a2947 [heise] Fix kaltura embeds extraction (closes #22514) 2019-09-26 23:11:02 +07:00
2a88a0c44d [ted] check for resources validity and extract subtitled downloads(closes #22513) 2019-09-26 11:44:57 +01:00
33c1c7d80f [youtube] Add support for owxfohz4kjyv25fvlqilyxast7inivgiktls3th44jhk3ej3i7ya.b32.i2p (#22292) 2019-09-25 02:43:34 +07:00
21d3c21e62 [nhk] Add support for clips 2019-09-25 02:39:25 +07:00
a373befa25 [nhk] fix video extraction(closes #22249)(closes #22353) 2019-09-24 20:24:17 +01:00
df63cafe49 [byutv] Fix extraction (refs #22070)
Downloading of new videos does not work due to DRM
2019-09-25 02:16:25 +07:00
d06daf23da [YoutubeDL] Honour all --get-* options with --flat-playlist (closes #22493) 2019-09-25 02:10:37 +07:00
8e9fdcbe27 [openload] Add support for oload.online (#22304) 2019-09-24 23:56:12 +07:00
666d808e70 [youtube] Add support for invidious.drycat.fr (#22451) 2019-09-24 23:16:46 +07:00
7d327fea5b [jwplatfom] do not match video URLs(#20596) (#22148) 2019-09-23 19:44:00 +00:00
4e3f1f0469 [youtube:playlist] Unescape playlist uploader (closes #22483) 2019-09-23 00:20:52 +07:00
4bc15a68d1 [bilibili] add support audio albums and songs(closes #21094) 2019-09-22 17:14:18 +01:00
edb2820ca5 [instagram] add support for tv URLs 2019-09-21 21:57:45 +01:00
6cf6b357f5 [mixcloud] allow uppercase letters in format urls(closes #19280) 2019-09-20 11:14:24 +01:00
f455a934e9 [brightcove] delegate all supported BrightcoveLegacyIE URLs to BrightcoveNewIE
closes #11523
closes #12842
closes #13912
closes #15669
closes #16303
2019-09-19 18:02:26 +01:00
d9d3098675 [hotstar] Use native HLS downloader by default 2019-09-19 03:03:07 +07:00
1cb812d3c2 [hotstar] Extract more formats (closes #22323) 2019-09-19 03:00:19 +07:00
6fd26a7d4a [9now] Fix extraction (closes #22361) 2019-09-19 02:31:39 +07:00
9cf26b6e1d [zdf] Bypass geo restriction 2019-09-19 01:11:52 +07:00
20e11b70ac [tv4] Fix extraction and extract series metadata (closes #22443) 2019-09-18 23:45:26 +07:00
e1f692f0b3 release 2019.09.12.1 2019-09-12 02:53:52 +07:00
2f851a7d7d [ChangeLog] Actualize
[ci skip]
2019-09-12 02:48:07 +07:00
4878759f3b [youtube] Remove quality and tbr for itag 43 (closes #22372) 2019-09-12 02:46:12 +07:00
303d3e142c [ChangeLog] Actualize
[ci skip]
2019-09-12 02:05:54 +07:00
bd10b229c0 release 2019.09.12 2019-09-12 01:21:21 +07:00
035c7a59e8 [ChangeLog] Actualize
[ci skip]
2019-09-12 01:18:25 +07:00
bf1317d257 [youtube] Quick extraction tempfix (closes #22367, closes #22163) 2019-09-11 22:44:47 +07:00
bff90fc518 [youtube] Add support for invidious tor instances (#22268) 2019-09-03 01:35:32 +07:00
31dbd054c8 [platzi] Improve client data extraction (closes #22290) 2019-09-03 01:24:20 +07:00
66d04c74e0 [platzi:course] Add support for authentication 2019-09-03 01:23:22 +07:00
d7da1e37c7 [nickjr] Add support for nickelodeonjunior.fr (#22246) 2019-09-02 00:59:57 +07:00
f620d0d860 release 2019.09.01 2019-09-01 03:33:02 +07:00
79dd8884bb [ChangeLog] Actualize
[ci skip]
2019-09-01 03:18:35 +07:00
df228355fd [xhamster:user] Add extractor (closes #16330, closes #18454) 2019-09-01 03:12:56 +07:00
8945b10f6e [xhamster] Add support for more domains 2019-09-01 03:09:04 +07:00
7cb51b5daf [extractor/generic] Improve squarespace detection and fix test (closes #21859, refs #21294, refs #21802) 2019-09-01 01:25:48 +07:00
d78657fd18 [extractor/generic] Add support for squarespace embeds (closes #21294) 2019-09-01 01:25:48 +07:00
cc73d5ad15 [openload] Fix domains regex 2019-09-01 01:25:48 +07:00
71f47617c8 [downloader/external] Respect mtime option for aria2c (#22242) 2019-09-01 00:24:43 +07:00
3f46a25a97 [verystream] add support for woof.tube (closes #22217) 2019-08-31 10:02:09 +01:00
9d058b3206 [dailymotion] Add support for lequipe.fr (closes #21328, closes #22152) 2019-08-29 23:08:19 +07:00
b500955a58 [openload] Add support for oload.vip (closes #22205) 2019-08-28 01:58:07 +07:00
Jay
acc86c9a97 [bbc] Fix some tests 2019-08-28 01:53:40 +07:00
Jay
b72305f078 [bbccouk] Extend _VALID_URL (closes #19200) 2019-08-28 01:53:40 +07:00
494d664e67 [youtube] Add support for invidious.nixnet.xyz and yt.elukerio.org (#22223) 2019-08-28 01:39:59 +07:00
d1fcf255c5 [safari] Fix authentication (closes #22161) (#22184) 2019-08-27 10:16:04 +07:00
183a18c4e7 [usanetwork] Fix extraction (closes #22105) 2019-08-26 03:38:54 +07:00
393cc31d5e [einthusan] Add support for einthusan.ca (#22171) 2019-08-21 09:52:59 +07:00
0add33abcb [youtube] Improve unavailable message extraction (refs #22117) 2019-08-16 23:44:11 +07:00
0326bcb6c1 [piksel] add subtitle capability (#20506) 2019-08-15 22:14:47 +00:00
def849e0e6 release 2019.08.13 2019-08-13 23:18:38 +07:00
69611a1616 [ChangeLog] Actualize
[ci skip]
2019-08-13 23:10:05 +07:00
351f37c022 [youtube:playlist] Improve flat extraction (closes #21927) 2019-08-13 05:02:52 +07:00
3bce4ff7d9 [downloader/fragment] Fix ETA calculation of resumed download (#21992) 2019-08-11 06:57:43 +07:00
ffddb11264 [YoutubeDL] check annotations availabilty(closes #18582) 2019-08-09 08:19:41 +01:00
64b6a4e91e [youtube] fix annotations extraction(closes #22045) 2019-08-09 08:16:53 +01:00
b3d39be239 [discovery] extract series meta field(#21808) 2019-08-08 23:23:58 +01:00
1357734978 [youtube] Improve error detection (#16445) 2019-08-06 02:32:44 +07:00
eb9c9c74a6 [vimeo] fix album extraction
closes #1933
closes #15704
closes #15855
closes #18967
closes #21986
2019-08-03 10:29:20 +01:00
5efbc1366f [roosterteeth] add support for watch URLs 2019-08-02 19:38:35 +01:00
995f319b06 [discovery] limit video data by show slug(closes #21980) 2019-08-02 18:08:26 +01:00
d9d3a5a816 [README.md] Move code from #21939 to the right place 2019-08-02 05:54:56 +07:00
4f2d735803 release 2019.08.02 2019-08-02 05:37:54 +07:00
2e9522b061 [ChangeLog] Actualize
[ci skip]
2019-08-02 05:36:32 +07:00
be306d6a31 [tvigle] Fix extraction and add support for HLS and DASH formats (closes #21967) 2019-08-02 05:25:01 +07:00
33b529fabd [yandexvideo] Add support for DASH formats (#21971) 2019-08-02 05:03:25 +07:00
07f3a05c87 [CONTRIBUTING.md] Add some more coding conventions (#21939) 2019-08-02 04:49:01 +07:00
535111657b [discovery] use API call for video data extraction(#21808) 2019-08-01 22:45:10 +01:00
826dcff99c Output batch filename when it could not be read (#21915) 2019-08-01 03:54:39 +07:00
9a37ff82f1 [mgtv] Extract format_note (#21881)
format_note should now show 标清, 高清, 超清, 蓝光, etc.
2019-08-01 03:45:02 +07:00
766c4f6090 [tvn24] Fix test 2019-07-31 02:32:45 +07:00
7279163412 [tvn24] Fix metadata extraction (closes #21833, closes #21834) 2019-07-31 02:32:45 +07:00
07ab44c420 [dlive] Relax _VALID_URL (#21909) 2019-07-31 01:43:49 +07:00
2c8b1a21e8 [openload] Add support for oload.best (#21913) 2019-07-31 01:40:50 +07:00
c2d125d99f [youtube] Improve metadata extraction for age gate content (closes #21943) 2019-07-31 00:14:33 +07:00
85c2c4b4ab release 2019.07.30 2019-07-30 09:43:47 +07:00
8614a03f9c [ChangeLog] Actualize
[ci skip]
2019-07-30 09:41:23 +07:00
8dbf751aa2 [youtube] improve title and description extraction(closes #21934) 2019-07-30 00:13:33 +01:00
90634acfcf release 2019.07.27 2019-07-27 03:44:55 +07:00
eaba9dd6c2 [ChangeLog] Actualize
[ci skip]
2019-07-27 03:43:33 +07:00
843ad1796b Fix typos (#21901) 2019-07-26 22:30:18 +07:00
608b8a4300 [yahoo:japannews] Add extractor (closes #21698) (#21265) 2019-07-22 00:59:36 +07:00
ab794a553c [ctsnews] PEP 8 2019-07-21 14:59:53 +07:00
3b446ab351 [discovery] add support go.discovery.com URLs 2019-07-20 20:20:53 +01:00
13a75688a5 [youtube] Fix some tests 2019-07-21 00:01:46 +07:00
2e18adec98 [youtube:playlist] Relax _VIDEO_RE (closes #21844) 2019-07-20 23:46:34 +07:00
9c1da4a9f9 [extractor/generic] Restrict --default-search schemeless URLs detection pattern (closes #21842) 2019-07-20 23:08:26 +07:00
5e1c39ac85 [extractor/common] Fix typo in thumbnails resolution description (#21817) 2019-07-17 22:47:53 +07:00
1824bfdcdf [vrv] fix CMS signing query extraction(closes #21809) 2019-07-16 22:51:10 +01:00
2f1991ff14 release 2019.07.16 2019-07-16 00:01:46 +07:00
8b4a0ebf10 [ChangeLog] Actualize
[ci skip]
2019-07-15 23:59:23 +07:00
f61496863d [asiancrush] Add support for yuyutv.com, midnightpulp.com and cocoro.tv (closes #21281, closes #21290) 2019-07-15 23:58:08 +07:00
799756a3b3 [kaltura] Check source format URL (#21290) 2019-07-15 23:58:08 +07:00
7d4dd3e5b4 [ctsnews] Fix YouTube embeds extraction (#21678) 2019-07-15 23:03:03 +07:00
f2a213d025 [einthusan] Add support for einthusan.com (closes #21748) (#21775) 2019-07-15 22:58:55 +07:00
791d2e8117 [youtube] Add support for invidious.mastodon.host (#21777) 2019-07-15 22:54:22 +07:00
2adedc477e [gfycat] Extend _VALID_URL (closes #21779) (#21780) 2019-07-15 22:53:20 +07:00
898238e9f8 [youtube] Restrict is_live extraction (closes #21782) 2019-07-14 20:30:05 +07:00
ce80cacefd release 2019.07.14 2019-07-14 03:10:49 +07:00
0250161c52 [yandexmusic] Add missing import 2019-07-14 03:09:16 +07:00
364a2cb658 [ChangeLog] Actualize
[ci skip]
2019-07-14 03:07:02 +07:00
2fe074a960 [porn91] Fix extraction (#21312) 2019-07-14 02:57:43 +07:00
c452790a79 [yandexmusic] Add support for multi disk albums and extract track number and disk number (closes #21420) (#21421)
* [yandexmusic] extract tracks from all volumes of an album (closes #21420)

* [yandexmusic] extract genre, disk_number and track_number

* [yandexmusic] extract decomposed artist names

* Update yandexmusic.py

* Update yandexmusic.py

* Update yandexmusic.py
2019-07-14 02:38:47 +07:00
d89a0a8026 [lynda] Handle missing subtitles (closes #20490, closes #20513) 2019-07-14 01:45:28 +07:00
ba036333bf [youtube] Add more invidious instances to _VALID_URL (#21694) 2019-07-14 01:23:22 +07:00
b7ef93f0ab [twitter] Improve uploader id extraction (closes #21705) 2019-07-14 01:19:17 +07:00
f9eeeda31c [spankbang] Fix and improve metadata extraction 2019-07-14 00:21:39 +07:00
5f562bd4bb [spankbang] Fix extraction (closes #21763, closes #21764) 2019-07-14 00:13:26 +07:00
b99f11a56b [dlive] restrict DLive Stream _VALID_URL regex 2019-07-13 14:11:57 +01:00
4a71ef6da6 [dlive] Add new extractor(closes #18080) 2019-07-13 13:08:19 +01:00
fd95105ed4 [livejournal] Add new extractor(closes #21526) 2019-07-13 12:47:02 +01:00
c72dc20d09 [roosterteeth] fix free episode extraction(#16094) 2019-07-13 10:13:07 +01:00
272355c172 [dbtv] fix extraction 2019-07-12 23:26:46 +01:00
57227618fe [spike] fix Bellator extraction 2019-07-12 22:50:37 +01:00
0441d6266c [rudo] remove extractor(closes #18430)(closes #18474)
Covered by generic extractor
2019-07-12 22:31:11 +01:00
82f68e4a01 [facebook] fallback to twitter:image meta for thumbnail extraction(closes #21224) 2019-07-12 22:02:06 +01:00
d4ece5d359 [bleacherreport] fix Bleacher Report CMS extraction 2019-07-12 21:56:49 +01:00
16d3672ad7 [espn] fix fivethirtyeight.com extraction 2019-07-11 23:37:34 +01:00
0dd58a523f [fivetv] relax video URL regex and support https URLs 2019-07-11 23:10:35 +01:00
27019dbb4b [youtube] Fix is_live extraction (closes #21734) 2019-07-12 03:45:58 +07:00
baf67a604d [youtube] Fix authentication (closes #11270) 2019-07-12 02:26:05 +07:00
0d1f4af39d release 2019.07.12 2019-07-12 00:43:54 +07:00
7612406bf9 [ChangeLog] Actualize
[ci skip]
2019-07-12 00:34:03 +07:00
4dcd4b7b16 [mgtv] Pass Referer HTTP header for format URLs (closes #21726) 2019-07-12 00:04:25 +07:00
5fc0896168 [beeg] Add support for api/v6 v2 URLs without t argument (closes #21701) 2019-07-11 23:37:09 +07:00
e4d53148f5 [funnyordie] move extraction to VoxMedia extractor and improve vox volume embed extraction(closes #16846) 2019-07-10 16:47:37 +01:00
cfe781d4fa [gameinformer] fix extraction(closes #8895)(closes #15363)(closes #17206) 2019-07-10 15:45:00 +01:00
253289656f [extractors] update funk.net import 2019-07-10 13:57:43 +01:00
4b30282616 [funk] fix extraction(closes #17915) 2019-07-10 13:54:49 +01:00
c9b0564ac1 [packtpub] Relax lesson _VALID_URL regex(closes #21695) 2019-07-09 11:56:16 +01:00
25d71fb058 [packtpub] fix extraction(closes #21268) 2019-07-09 08:28:56 +01:00
a6389abfd7 [philharmoniedeparis] Relax _VALID_URL (closes #21672) 2019-07-06 23:17:45 +07:00
d18003a141 [peertube] Detect embed URLs in generic extraction (closes #21666) 2019-07-06 00:50:56 +07:00
d1850c1a97 [mixer:vod] Relax _VALID_URL (closes #21657) (#21658) 2019-07-05 22:47:32 +07:00
c9fa84d88e [lecturio] add support id based URLs(closes #21630) 2019-07-04 15:59:45 +01:00
a30c2f4055 [go] Add site info for disneynow (closes #21613) 2019-07-04 04:01:30 +07:00
5ae9b8b3a3 [adobepass] Add support for AT&T U-verse (mso ATT) (closes #13938, closes #21016) 2019-07-04 03:57:11 +07:00
cdb7c7d147 [ted] Restrict info regex (closes #21631) 2019-07-04 02:04:23 +07:00
2da4316e48 [twitch:vod] Actualize m3u8 URL (#21538, #21607) 2019-07-03 23:22:23 +07:00
313877c6a2 [vzaar] Fix videos with empty title (closes #21606) 2019-07-03 23:16:40 +07:00
e61ac1a09c [tvland] fix extraction(closes #21384) 2019-07-03 13:31:47 +01:00
ff0f4cfeba [arte] clean extractor(closes #15583)(closes #21614) 2019-07-02 22:09:40 +01:00
1335bf10f6 release 2019.07.02 2019-07-02 01:09:59 +07:00
c8343f0a43 [ChangeLog] Actualize
[ci skip]
2019-07-02 01:07:54 +07:00
d1e4116427 [vevo] Add support for embed.vevo.com URLs (#21565) 2019-07-02 00:13:23 +07:00
9baf69af45 [openload] Add support for oload.biz (#21574) 2019-07-02 00:11:38 +07:00
Fai
918398092c [xiami] Update API base URL (#21575) 2019-07-02 00:10:55 +07:00
4e2491f066 [yourporn] Fix extraction (#21585) 2019-07-02 00:05:51 +07:00
976e1ff7f9 [acast] add support for URLs with episode id(closes #21444) 2019-07-01 12:05:18 +01:00
5e3da0d42b [dailymotion] add support embed with DM.player js call 2019-07-01 08:37:21 +01:00
c560680247 [soundcloud] Update client id 2019-06-29 00:33:35 +07:00
f7a147e3b6 [utils] Introduce random_user_agent and use as default User-Agent (closes #21546) 2019-06-29 00:32:43 +07:00
8c8cae91ec release 2019.06.27 2019-06-27 23:57:33 +07:00
232331c0d2 [ChangeLog] Actualize
[ci skip]
2019-06-27 23:55:15 +07:00
4f71473ef1 [go] Add support for disneynow.com (closes #21528) 2019-06-27 22:59:30 +07:00
6625bf200d [mixer:vod] Relax _VALID_URL (closes #21531) (#21536) 2019-06-27 22:24:46 +07:00
f562994660 [drtv] Relax _VALID_URL 2019-06-27 22:18:10 +07:00
509bcec37b [fusion] fix extraction(closes #17775)(closes #21269) 2019-06-27 12:06:31 +01:00
1d83e9bd4b [nfb] Remove extractor (closes #21518)
Covered by generic extractor
2019-06-25 00:12:31 +07:00
27cef8885d [beeg] Add support for api/v6 v2 URLs (closes #21511) 2019-06-24 23:01:52 +07:00
3031b7c4ed [brightcove:new] Add support for playlists (#21331) 2019-06-23 17:04:05 +07:00
695720ebe8 [openload] Add support for oload.life (#21495) 2019-06-23 04:31:43 +07:00
2605043d6d [vimeo:channel,group] Make title extraction no fatal 2019-06-23 02:16:09 +07:00
091c9b4316 [vimeo:likes] Implement extrator in terms of channel extractor
This allows to obtain videos' ids before extraction (#21493)
2019-06-23 02:13:46 +07:00
9634de178d [pornhub] Add support for more paged video sources 2019-06-22 08:37:07 +07:00
1f7a563ab0 [pornhub] Add support for downloading single pages and search pages (closes #15570) 2019-06-22 06:01:43 +07:00
21b08463a7 [pornhub] Rework extractors (closes #11922, closes #16078, closes #17454, closes #17936) 2019-06-22 05:34:46 +07:00
31ce6e9966 [youtube] Add another signature function pattern 2019-06-22 02:22:41 +07:00
1c11204056 [tf1] Improve extraction and fix issues (closes #21372) 2019-06-22 00:40:06 +07:00
9c2aaac268 [tf1] Fix wat id extraction (closes #21365) 2019-06-22 00:40:00 +07:00
d415957dbc [crunchyroll] Move Accept-Language workaround to video extractor since it causes playlists not to list any videos 2019-06-22 00:15:52 +07:00
4681441d2f [crunchyroll:playlist] Fix and relax title extraction (closes #21291, closes #21443) 2019-06-22 00:07:26 +07:00
9842d29d66 release 2019.06.21 2019-06-21 23:04:09 +07:00
bc6438c092 [ChangeLog] Actualize
[ci skip]
2019-06-21 23:01:09 +07:00
abefc03f51 [youtube] Update signature function patterns (closes #21469, closes #21476) 2019-06-21 22:58:42 +07:00
c40714cdee [youtube] Make --write-annotations non fatal (closes #21452) 2019-06-20 00:57:58 +07:00
7c24a58bdb [sixplay] add support for rtlmost.hu(#21405) 2019-06-16 06:32:47 +01:00
b85eae0f05 [youtube] Hardcode codec metadata for av01 video only formats (closes #21381) 2019-06-14 01:59:05 +07:00
28cc2241e4 [utils] Restrict parse_codecs and add theora as known vcodec (#21381) 2019-06-14 01:56:17 +07:00
8361e7f934 [toutv] update client key(closes #21370) 2019-06-12 21:41:46 +01:00
427cc21531 [biqle] remove unnecessary regex group 2019-06-10 15:17:26 +01:00
f991dd2bfb [biqle] add support for new embed domain 2019-06-10 14:58:37 +01:00
35c2dd48d9 [cbs] improve DRM protected videos detection(closes #21339) 2019-06-09 00:13:07 +01:00
3b2fd09596 release 2019.06.08 2019-06-08 03:47:30 +07:00
e35fc5ebc7 [ChangeLog] Actualize
[ci skip]
2019-06-08 03:45:31 +07:00
dbb1886114 [ted] Improve playlist extraction (closes #21032) 2019-06-08 03:07:08 +07:00
c2ee6fa66a [ted] Fix playlist extraction (closes #20844) 2019-06-08 03:07:02 +07:00
4831ef7fe4 [vlive] Add comments to tests and quotes consistency 2019-06-08 02:42:01 +07:00
178663df52 [vlive:playlist] Redirect to video extractor when no playlist is found (#20590) 2019-06-08 02:40:47 +07:00
ef19739e64 [vlive] Improve and style (closes #21209) 2019-06-08 02:22:23 +07:00
01b517a20a [vlive] Add CH+ support (closes #16887) 2019-06-08 02:22:15 +07:00
f4c99cd635 [openload] Add support for oload.website (#21329) 2019-06-08 02:03:58 +07:00
e75220b11a [tvnow] Extract HD formats (closes #21201) 2019-06-08 00:45:28 +07:00
2efefddafd [redbulltv:rrn:content] Add extractor (closes #21297) 2019-06-07 23:48:57 +07:00
bf3c932663 [youtube] Fix average rating extraction (closes #21304) 2019-06-07 23:13:33 +07:00
4c78c3d700 [bitchute] Extract HTML5 formats (closes #21306) 2019-06-07 22:58:19 +07:00
2e11e51c04 [cbsnews] fix extraction(closes #9659)(closes #15397) 2019-06-05 01:23:23 +01:00
1a01639bf9 [downloader/common] Improve rate limit (#21301) 2019-06-05 03:06:35 +07:00
59ca17b1c8 [vvvvid] relax _VALID_URL(closes #21299) 2019-06-04 21:03:52 +01:00
c94c121a99 Revert "[southpark] add support for videos with english audio track"
This reverts commit 0c84002650.
2019-06-04 00:37:58 +01:00
0c84002650 [southpark] add support for videos with english audio track(closes #21271) 2019-06-03 12:43:39 +01:00
c5eb75b35a [prosiebensat1] add support for new API(closes #21272) 2019-06-01 15:12:30 +01:00
0e2dd3fcbc [vrv] extract adaptive_hls formats(closes #21243) 2019-06-01 11:17:44 +01:00
26a87972a9 [viki] Switch to HTTPS (closes #21001) 2019-05-29 04:42:19 +07:00
33b2218b2f [LiveLeak] Check if the original videos exist (closes #21206) (#21208) 2019-05-28 12:31:11 +00:00
ead467a9c1 [rtp] fix extraction(closes #15099) 2019-05-28 04:58:12 +01:00
0d29751890 [youtube] improve DRM protected videos detection(#1774) 2019-05-25 23:15:59 +01:00
25b83c2a0e [srgssrplay] Improve _VALID_URL (closes #21155) 2019-05-24 00:45:06 +07:00
3fe774722b [srgssrplay] Add support for popupvideoplayer URLs 2019-05-24 00:44:37 +07:00
f4cc2ca503 [24video] Add support for porno.24video.net (closes #21194) 2019-05-24 00:38:06 +07:00
11ec06de7f [24video] Add support for 24video.site (closes #21193) 2019-05-24 00:35:46 +07:00
53cd37bac5 [utils] Improve strip_or_none 2019-05-24 00:03:01 +07:00
f856816b94 [extractor/common] Strip src attribute for HTML5 entries code (closes #18485, closes #21169) 2019-05-23 23:52:11 +07:00
8af49fc276 [pornflip] Remove extractor 2019-05-23 23:48:06 +07:00
9c5f2988b9 [criterion] Remove extractor (closes #21195) 2019-05-23 23:38:01 +07:00
afd4985f72 [travis] Force dist to Ubuntu Trusty by default
According to https://blog.travis-ci.com/2019-04-15-xenial-default-build-environment Ubuntu Xenial is now default, but it lacks python 2.6, 3.2 and 3.3 support needed by tests
2019-05-23 06:06:49 +07:00
2c53c0ebc6 [pornhub] Use https (#21061) 2019-05-23 03:56:54 +07:00
bbf1defe58 [bitchute] Fix uploader extraction (#21076) 2019-05-23 03:51:50 +07:00
186d185b6e [streamcloud] Reduce waiting time to 6 seconds (#21092) 2019-05-23 03:46:20 +07:00
612300a686 [novamov] Remove extractors (#21077)
Sites no longer exist
2019-05-23 03:38:48 +07:00
ea75382094 [openload] Add support for oload.press (#21135) 2019-05-23 03:30:17 +07:00
e438e81469 [vivo] Improve extraction (closes #19217) 2019-05-23 03:06:07 +07:00
0e6f914b3b [vivo] Fix extraction (closes #18906) 2019-05-23 03:06:07 +07:00
6ab30ff50b release 2019.05.20 2019-05-20 23:29:49 +07:00
42c971341b [ChangeLog] Actualize
[ci skip]
2019-05-20 23:24:27 +07:00
ce2fe4c01c [extractor/common] Add doc string for _apply_first_set_cookie_header 2019-05-20 23:23:18 +07:00
a9e03736df [safari] Fix authentication (closes #21090) 2019-05-18 03:33:40 +07:00
e3c1266f49 [extractor/common] Move workaround for applying first Set-Cookie header into a separate method 2019-05-18 03:17:15 +07:00
82e91d20a0 [vrt] fix extraction(closes #20527) 2019-05-15 15:02:51 +01:00
170d644440 [canvas] add support for vrtnieuws and sporza site ids and extract AES HLS formats 2019-05-15 14:58:57 +01:00
5831742840 [vrv] extract captions(closes #19238) 2019-05-15 10:39:59 +01:00
a277dd33eb [tele5] Improve video id extraction 2019-05-13 00:13:00 +07:00
0e0bfd334c [tele5] Relax _VALID_URL (closes #21020, closes #21063) 2019-05-13 00:11:48 +07:00
e6a25fea23 [svtplay] Update API URL (closes #21075) 2019-05-12 23:24:01 +07:00
b7df8f90a7 [yahoo:gyao] add X-User-Agent header to dam proxy requests(closes #21071) 2019-05-12 10:23:36 +01:00
8ae32e1882 release 2019.05.11 2019-05-11 06:07:28 +07:00
07a2cef782 [ChangeLog] Actualize
[ci skip]
2019-05-11 06:04:53 +07:00
8ed7a23328 [extractor/common] Fix typo 2019-05-11 04:53:48 +07:00
3089bc748c Fix W504 and disable W503 (closes #20863) 2019-05-11 03:57:40 +07:00
d23e85515a [tvnow] Fix W605 2019-05-11 03:57:40 +07:00
cd37ef44e9 [commonmistakes] Fix E117 2019-05-11 03:57:40 +07:00
169f8d0fe1 [cloudflarestream] Add support for videodelivery.net (#21049) 2019-05-11 03:09:00 +07:00
0db2b275dd [byutv] Improve extraction and update DVR test (closes #20676) 2019-05-11 03:06:18 +07:00
ab11674502 [byutv] Add support for DVR videos (closes #20574)
Fix code style on brackets (flake8)

Add more information to test info_dict
2019-05-11 03:06:12 +07:00
68b92aa1b4 [gfycat] Add support for URLs with tags (closes #20696) (#20731) 2019-05-11 02:12:45 +07:00
fd35d8cdfd [utils] Transliterate "þ" as "th" (#20897)
Despite visual similarity "þ" is unrelated to "p".
It is normally transliterated as "th":

    $ echo þ-Þ | iconv -t ASCII//TRANSLIT
    th-TH
2019-05-11 01:42:31 +07:00
4eec112740 [openload] Add support for verystream.com (closes #20701) (#20967) 2019-05-11 01:35:56 +07:00
027ffdca0d [youtube] Use sp field value for signature field name (closes #18841, closes #18927, closes #21028) 2019-05-10 08:38:05 +07:00
a5b92d3590 [yahoo:gyao] extend _VALID_URL(closes #21008) 2019-05-09 04:14:19 +01:00
b45a9e698e [youtube] Fix channel id extraction (closes #20982) (#21003) 2019-05-09 00:58:47 +07:00
71ebd35d50 [sky] add support for news.sky.com (closes #13055) 2019-05-07 10:17:18 +01:00
f8c55c6664 [youtube:entrylistbase] Retry on 5xx HTTP errors (#20965) 2019-05-06 01:25:16 +07:00
876fed6bf3 [francetvinfo] Extend video id extraction (closes #20619) (#20740) 2019-05-05 06:26:30 +07:00
c9856648db [4tube] Update token hosts (closes #20918) 2019-05-04 03:26:40 +07:00
2533f5b691 [hotstar] Move to API v2 (closes #20931) 2019-05-04 03:11:25 +07:00
e0dde1d8e2 [fox] fix Uplynk PrePlay error handling under python 2(#20925) 2019-05-02 10:46:29 +01:00
62d10f0d32 [fox] Fix API error handling under python 2 (closes #20925) 2019-05-02 00:36:52 +07:00
274519dd08 [redbulltv] Extend _VALID_URL (closes #20922) 2019-05-01 21:36:19 +07:00
091200c368 release 2019.04.30 2019-04-30 06:11:50 +07:00
67bfbe4942 [ChangeLog] Actualize
[ci skip]
2019-04-30 06:08:51 +07:00
54f3b61216 [openload] Use real Chrome versions (closes #20902) 2019-04-30 05:59:12 +07:00
a61ce71468 [youtube] Remove info el for get_video_info request
Since it does not work for quite a long time
2019-04-30 04:49:12 +07:00
026fbedc85 [youtube] Improve extraction robustness
Fail on missing token only when no formats found
2019-04-30 04:32:55 +07:00
6e07b5a6d5 [dramafever] Remove extractor(closes #20868) 2019-04-28 18:02:41 +01:00
c464e1df2c [adn] fix subtitle extraction(#12724) 2019-04-28 17:50:47 +01:00
92bc97d398 [youtube] extract album from Music in this video section(#20301) 2019-04-28 17:38:20 +01:00
f916abc0ac [ccc] Improve extraction (closes #14601, closes #20355) 2019-04-28 23:08:09 +07:00
24510bdcfa [ccc] Extract creator 2019-04-28 23:07:41 +07:00
ae8c13565e [ccc:playlist] Add extractor 2019-04-28 23:07:01 +07:00
280913800d [sverigesradio] improve extraction(closes #18635) 2019-04-28 12:03:39 +01:00
7ff8ad80f1 [sverigesradio] Add extractor 2019-04-28 10:07:06 +01:00
4e4db743e7 [cinemax] Add new extractor 2019-04-28 00:42:55 +01:00
3545d38bfb [sixplay] add missing parenthesis 2019-04-27 10:32:53 +01:00
2309d6bf92 [sixplay] try to extract non drm protected manifests(closes #20849) 2019-04-27 10:17:34 +01:00
822b9d9cb0 [youtube] improve Youtube Music Auto-generated description parsing(closes #20742) 2019-04-27 09:16:17 +01:00
5caabd3c70 [youtube] Extract additional meta data from video description on youtube music videos
YouTube music videos often have auto-generated video descriptions that can be
utilized to extract additional information about the video. This is desirable
in order to provide the user with as much meta data as possible. This commit
adds extraction methods for the following fields for youtube music videos:
- artist (fallback extraction methods added)
- track (fallback extraction methods added)
- album (new in this commit)
- release_date (new in this commit)
- release_year (new in this commit)

4 tests have been added to test this new functionality:
- YoutubeIE tests 27, 28, 29, and 30

Resolves: #20599
2019-04-27 09:09:54 +01:00
aa05a093bb [wrzuta] Remove extractor (closes #20684) (#20801)
Wrzuta.pl was shut down in 2017.
2019-04-27 05:12:15 +07:00
60e67c5b2c [twitch] Prefer source format (closes #20850) 2019-04-27 05:08:27 +07:00
eefa0f2157 Move issue template templates into separate folder 2019-04-27 04:55:30 +07:00
6f366ef30c Issue template overhaul 2019-04-27 04:50:47 +07:00
88b547492f [twitcasting] Add support for private videos (#20843) 2019-04-26 16:17:40 +00:00
00a9a25cf9 [twitcasting] Fix test: video title (#20840) 2019-04-26 09:34:23 +00:00
97abf05ad3 [reddit] check thumbnail URL(closes #20030) 2019-04-26 10:26:51 +01:00
da668a23bd [ISSUE_TEMPLATE.md] Add entry on argument escaping in make-sure checklist
[ci skip]
2019-04-26 00:46:41 +07:00
58ef5e7881 [yandexmusic] fix track url extraction(closes #20820) 2019-04-25 11:36:44 +01:00
3e7ec5330a release 2019.04.24 2019-04-24 10:05:54 +07:00
98933c14e1 [ChangeLog] Actualize
[ci skip]
2019-04-24 10:05:08 +07:00
56667d622c [youtube] Fix extraction (closes #20758, closes #20759, closes #20761, closes #20762, closes #20764, closes #20766, closes #20767, closes #20769, closes #20771, closes #20768, closes #20770) 2019-04-24 09:58:00 +07:00
50d660479d [toutv] fix extraction and extract series info(closes #20757) 2019-04-24 00:28:00 +01:00
1fa8893734 [vrv] add support for movie listings(closes #19229) 2019-04-22 23:50:37 +01:00
15be3eb5e5 [youtube] raise ExtractorError when no data available(#20737) 2019-04-22 20:52:43 +01:00
e09965d550 [soundcloud] Add support for new rendition and improve extraction (closes #20699) 2019-04-23 00:39:36 +07:00
3fd86cfe13 [ooyala] add support for geo verification proxy 2019-04-22 10:04:56 +01:00
fdc2183650 [nrl] Add new extractor(closes #15991) 2019-04-22 10:04:00 +01:00
85b6335d55 [vimeo] extract live archive source format(#19144) 2019-04-21 21:05:58 +01:00
c25720ef6a [vimeo] add support live streams and improve info extraction(closes #19144) 2019-04-21 17:20:52 +01:00
c9b19d7a55 [ntvcojp] Add new extractor 2019-04-21 14:51:26 +01:00
47cfa00516 [nhk] extract rtmpt format 2019-04-21 13:25:04 +01:00
061d1cd948 [nhk] add support for audio URLs 2019-04-21 13:17:22 +01:00
5de538787d [udemy] add another course id extraction pattern(closes #20491) 2019-04-19 20:44:59 +01:00
9abeefd527 [openload] Add support for oload.services (closes #20691) 2019-04-18 23:56:20 +07:00
f3914b06a0 [openload] Add support for openloed.co (closes #20691)
While the .co could be captured directly, I anticipate that there will be more TLD's for openloed in the future.
2019-04-18 01:51:32 +07:00
81d989c21e [bravotv] fix extraction(closes #19213) 2019-04-18 01:50:30 +07:00
cd6c75b05f release 2019.04.17 2019-04-18 01:50:25 +07:00
9846935256 [ChangeLog] Actualize
[ci skip]
2019-04-17 00:15:48 +07:00
7fc3b68ad3 [openload] Randomize User-Agent (closes #20688) 2019-04-17 00:08:50 +07:00
c4341ea47e [openload] Add support for oladblock domains (#20471) 2019-04-16 23:50:04 +07:00
e6c9ae31df [adn] fix subtitle extraction(#12724) 2019-04-16 13:04:13 +01:00
6104cc1591 [openload] add test for oladblock.me 2019-04-15 23:09:05 -04:00
f114e43d38 [openload] add oladblock.me domain 2019-04-15 23:08:28 -04:00
cb6cd76f7b [aol] add support for localized websites 2019-04-14 23:18:36 +01:00
0b758fea1c [yahoo] add support GYAO episode URLs 2019-04-14 15:01:01 +01:00
3534b6329a [yahoo] add support for streaming.yahoo.co.jp(closes #5811)(closes #7098) 2019-04-14 14:39:20 +01:00
174f62992d [yahoo] add support for gyao.yahoo.co.jp 2019-04-14 14:29:04 +01:00
1038532213 [aenetworks] add encoding declaration 2019-04-14 13:18:16 +01:00
4f1e02ad60 [aenetworks] fix history topic extraction and extract more formats 2019-04-14 11:46:33 +01:00
180a9dff1f [cbs] extract smpte and vtt subtitles 2019-04-13 17:02:22 +01:00
972d2dd0bc [streamango] add support for streamcherry.com (#20592) 2019-04-13 15:05:24 +07:00
11edb76610 [README.md] Rephrase usage example comment (#20614) 2019-04-13 15:03:42 +07:00
8721b09751 [yourporn] Add support for sxyprn.com (#20646) 2019-04-13 15:02:09 +07:00
dc27fd8bb8 [mgtv] fix extraction(closes #20650) 2019-04-12 09:19:09 +01:00
c912029480 [linkedin:learning] use urljoin for form action url(closes #20431) 2019-04-11 08:44:58 +01:00
118f7add3b [gdc] add support for kaltura embeds and update tests(closes #20575) 2019-04-09 11:23:47 +01:00
4bc12b8f81 [dispeak] improve mp4 bitrate extraction 2019-04-09 11:21:46 +01:00
5ca3459828 [kaltura] sanitize embed URLs 2019-04-09 11:20:26 +01:00
9c017253e8 [jwplatfom] do not match manifest URLs(#20596) 2019-04-08 16:34:03 +01:00
9045d28b5e [aol] restrict url regex and improve format extraction 2019-04-07 21:31:26 +01:00
7c2ecbc1cc [tiktok] Add support for new URL schema (closes #20573) 2019-04-07 21:06:09 +07:00
d562cac9dc [stv:player] Add new extractor(closes #20586) 2019-04-07 12:40:14 +01:00
9ed06812ec [streamango] add support for streamcherry.com 2019-04-06 23:59:41 -04:00
bf6fb8b9dc [openload] add tests 2019-04-06 23:38:40 -04:00
a46d9e5b41 release 2019.04.07 2019-04-07 04:19:46 +07:00
aa5338118e [ChangeLog] Actualize
[ci skip]
2019-04-07 04:16:45 +07:00
8410653f24 [ruutu] Add support for audio podcasts (closes #20473, closes #20545) 2019-04-07 03:18:10 +07:00
f4da808036 [xvideos] Extract all thumbnails (closes #20432) 2019-04-07 02:59:09 +07:00
f412970164 [README.md] Fix lists formatting (closes #20558)
Lists have to be separated from the previous paragraph by a blank line
in certain variants of Markdown, otherwise they are not interpreted as
lists.

This change ensures that that the youtube-dl.1 man page, which is
generated from README.md with the help of pandoc, is formatted
correctly.
2019-04-07 02:28:31 +07:00
059cd768b9 [vk] Remove unused import 2019-04-07 02:17:54 +07:00
c701472fc9 [platzi] Add extractor (closes #20562) 2019-04-07 02:15:52 +07:00
19591facea [dvtv] remove unnecessary comments and spaces 2019-04-06 16:45:11 +01:00
b9aad6c427 [dvtv] Fix extraction (closes #18514) 2019-04-06 16:37:07 +01:00
9f182c23ba [vrv] add basic support for individual movie links(#19229) 2019-04-06 09:22:25 +01:00
4810655cd6 [bfi:player] Add new extractor(#19235) 2019-04-05 19:35:35 +01:00
a7978f8e2a [hbo] fix extraction and extract subtitles(closes #14629)(closes #13709) 2019-04-05 18:08:43 +01:00
19041a3877 [youtube] extract srv[1-3] subtitle formats(#20566) 2019-04-05 16:18:57 +01:00
afb7496416 [adultswim] fix extraction(closes #18025) 2019-04-05 11:45:49 +01:00
69e6efac16 [teamcoco] fix extraction and add suport for subdomains(closes #17099)(closes #20339) 2019-04-05 08:26:04 +01:00
2bbde1d09a [adn] fix subtitle compatibility with ffmpeg 2019-04-04 17:59:20 +01:00
b966740cf7 [adn] fix extraction and add support for positioning styles(closes #20549) 2019-04-04 14:50:16 +01:00
220828f2d6 [vk] use a more unique video id(closes #17848) 2019-04-03 11:08:42 +01:00
977a782110 [rtl2] update player_url 2019-04-03 10:20:01 +01:00
a2b6f946f1 [newstube] fix extraction 2019-04-03 10:19:36 +01:00
4f7db46887 [rtl2] improve _VALID_URL regex 2019-04-03 01:00:02 +01:00
d7d86fdd49 [download/external] pass rtmp_conn to ffmpeg 2019-04-02 22:41:23 +01:00
f8987163fb [adobeconnect] Add new extractor(closes #20283) 2019-04-02 22:40:39 +01:00
313e8b2b18 [gaia] add support for authentication(closes #14605) 2019-04-02 15:50:06 +01:00
c0b7d11713 [YoutubeDL] Add ffmpeg_location to post processor options (closes #20532) 2019-04-02 01:29:44 +07:00
efee62ac7f [mediasite] Add support for dashed ids and named catalogs (closes #20531) 2019-04-02 01:13:52 +07:00
38287d251d release 2019.04.01 2019-04-01 23:55:17 +07:00
25d9243141 [ChangeLog] Actualize
[ci skip]
2019-04-01 23:53:28 +07:00
93bb6b1bae [weibo] Extend _VALID_URL (#20496) 2019-03-31 01:31:33 +07:00
b43c5f474a [xhamster] Add support for xhamster.one (closes #20508) 2019-03-31 01:27:45 +07:00
4014a48622 [mediasite:catalog] Add extractor (closes #20507) 2019-03-31 01:21:53 +07:00
99fe330070 [teamtreehouse] Add new extractor(closes #9836) 2019-03-28 16:55:57 +01:00
c4c888697e [ina] add support for audio URLs 2019-03-27 18:49:29 +01:00
b27a71e66c [ina] improve extraction 2019-03-27 18:29:24 +01:00
de74ef83b7 [cwtv] fix episode number extraction(closes #20461) 2019-03-27 18:01:51 +01:00
cf3d399727 [openload] add support for oladblock.services and oladblock.xyz domains 2019-03-25 12:04:31 -04:00
8cb10807ed [npo] Improve DRM detection 2019-03-23 21:43:50 +07:00
b8526c78f9 [pornhub] Add support for DASH formats (closes #20403) 2019-03-23 01:09:33 +07:00
5e1271c56d [utils] Improve int_or_none and float_or_none (#20403) 2019-03-23 01:08:54 +07:00
050afa60c6 Check for valid --min-sleep-interval when --max-sleep-interval is specified 2019-03-21 22:55:03 +07:00
c4580580f5 [svtplay] Update API endpoint (closes #20430) 2019-03-21 22:39:35 +07:00
0a8e251b35 release 2019.03.18 2019-03-18 01:36:41 +07:00
2ed2ebdb36 [ChangeLog] Actualize
[ci skip]
2019-03-18 01:33:37 +07:00
8428fdccf2 [yandexvideo] Add extractor 2019-03-17 17:33:07 +07:00
04988b55b5 [openload] Improve embed detection 2019-03-17 15:34:21 +07:00
034f5fb5ee [radiocanada:audiovideo] Fix typo 2019-03-17 14:34:48 +07:00
a63782b581 [corus] Add support for bigbrothercanada.ca (#20357) 2019-03-17 14:20:21 +07:00
0146c6cde6 [orf:radio] Extract series (#20012) 2019-03-17 09:57:02 +07:00
ddff25c5d1 [extractors] Remove superfluous whitespace 2019-03-17 09:41:16 +07:00
81dada0b4b [cbc:watch] Add support for gem.cbc.ca (closes #20251, #20359) 2019-03-17 09:37:29 +07:00
eba3a2f9ef [anysex] Remove extractor (closes #19279) 2019-03-17 09:15:24 +07:00
d493f15c11 [extractor/common] Improve HTML5 entries extraction and add some realworld tests 2019-03-17 09:09:32 +07:00
0dc41787af [utils] Introduce parse_bitrate 2019-03-17 09:07:47 +07:00
e5cfb779ea [ciscolive] Add support for new URL schema (closes #20320, #20351) 2019-03-17 07:18:54 +07:00
77d95677b7 [youtube] Add support for invidiou.sh (#20309) 2019-03-17 07:15:15 +07:00
ffbd1368df [update] Hide update URLs behind redirect 2019-03-17 07:08:20 +07:00
2e27421c70 [test_InfoExtractor] Add test for #20346 2019-03-15 01:20:24 +07:00
79d2077edc [extractor/common] Fix url meta field for unfragmented DASH formats (closes #20346) 2019-03-15 00:42:14 +07:00
6db03a29d1 [anitube] Remove extractor
site no longer exists
2019-03-13 10:47:55 +01:00
47f9792620 [ruleporn] Remove extractor (closes #15344, closes #20324)
Covered by generic extractor
2019-03-12 22:55:32 +07:00
68867668cf [npr] fix extraction(closes #10793)(closes #13440) 2019-03-12 16:52:28 +01:00
e7e3ec828b [biqle] fix extraction(closes #11471)(closes #15313) 2019-03-12 08:08:54 +01:00
e585547284 [viddler] Modernize 2019-03-11 15:53:44 +01:00
4993132344 [pandatv] fix condition syntax 2019-03-11 15:37:19 +01:00
8569058f88 [moevideo] fix extraction 2019-03-11 15:18:09 +01:00
612a159510 [primesharetv] remove extractor 2019-03-11 09:51:26 +01:00
f517c0c0a7 [hypem] Modernize and extract more metadata(closes #15320) 2019-03-11 09:43:06 +01:00
197cf8b794 [veoh] fix extraction 2019-03-11 09:04:06 +01:00
fddb4a3a8b [escapist] Modernize 2019-03-11 08:18:57 +01:00
99534feaff [videomega] remove extractor(closes #10108) 2019-03-11 07:27:57 +01:00
cdc7baffca [beeg] Add support for beeg.porn (closes #20306) 2019-03-11 04:44:40 +07:00
067aa17edf Start moving to ytdl-org 2019-03-11 04:00:54 +07:00
2765503713 [vimeo:review] improve config url extraction and extract original format(closes #20305) 2019-03-10 15:03:32 +01:00
0d08bcdb70 [fox] detect geo restriction and authentication errors(#20208) 2019-03-10 09:37:40 +01:00
10734553fe release 2019.03.09 2019-03-09 02:53:18 +07:00
bba35695eb [ChangeLog] Actualize
[ci skip]
2019-03-09 02:52:08 +07:00
9d74ea6d36 [francetv:site] Relax video id regex and update test (closes #20268) 2019-03-08 23:28:24 +07:00
7b6e760870 [toutv] detect invalid login error 2019-03-06 09:28:14 +01:00
829685b88a [toutv] fix authentication(closes #20261) 2019-03-06 09:20:39 +01:00
fca9baf0da [test] Fix test_compat_etree_Element 2019-03-06 02:46:26 +07:00
d347b52b63 [urplay] Extract timestamp (#20235) 2019-03-06 02:11:32 +07:00
97157c692c [openload] Add support for oload.space 2019-03-06 01:34:34 +07:00
a551768acf [facebook] Improve uploader extraction (closes #20250) 2019-03-06 01:27:22 +07:00
ee0ba927aa Use compat_etree_Element 2019-03-06 01:21:57 +07:00
399f76870d [compat] Introduce compat_etree_Element 2019-03-06 01:18:52 +07:00
e5ada4f3ad [extractor/common] Fallback url to base URL for DASH formats 2019-03-06 00:33:08 +07:00
bb6f112d9d [npo] Improve ISM extraction 2019-03-05 23:58:46 +07:00
c17eb5b4b0 [rai] Improve extraction (closes #20253) 2019-03-05 23:56:42 +07:00
d9eb580a79 [extractor/common] Do not fail on invalid data while parsing F4M manifest in non fatal mode 2019-03-05 23:56:33 +07:00
5dcd630dca [paramountnetwork] fix mgid extraction(closes #20241) 2019-03-04 22:26:55 +01:00
c790e93ab5 [extractor/common] Clarify url and manifest_url meta fields 2019-03-05 00:41:53 +07:00
39c780fdec [extractor/common] Return MPD manifest as format's url meta field (#20242)
For symmetry with other segmented media
2019-03-05 00:40:57 +07:00
e7e62441cd [utils] strip #HttpOnly_ prefix from cookies files (#20219) 2019-03-03 19:23:59 +07:00
0a5baf9c21 [libsyn] improve extraction(closes #20229) 2019-03-03 06:18:51 +01:00
8ae113ca9d [youtube] Add more invidious instances
See [Invidious-Instances](https://github.com/omarroth/invidious/wiki/Invidious-Instances) for the reference.
2019-03-03 08:19:36 +07:00
7aeb788e56 [travis] Remove sudo: false
Travis now recommends removing `sudo: false` from configuration: https://blog.travis-ci.com/2018-11-19-required-linux-infrastructure-migration.
2019-03-03 08:16:48 +07:00
7465e0aee2 [spankbang] Fix extraction (closes #20023) 2019-03-03 06:25:45 +07:00
a8f83f0c56 [crunchyroll] fix is_logged check 2019-03-02 08:25:47 +01:00
dca0e0040a Revert "use older login method(closes #11572)"
This reverts commit cc6a960e13.
2019-03-02 08:01:42 +01:00
398e1e21d6 [espn] extend _VALID_URL regex(closes #20013) 2019-03-01 15:34:05 +01:00
c5b02efe20 [sixplay] handle videos with empty assets(closes #20016) 2019-03-01 15:08:11 +01:00
06242d44fe [vimeo] add support for Vimeo Pro portfolio protected videos(closes #20070) 2019-03-01 08:14:34 +01:00
04c33bdfb3 release 2019.03.01 2019-03-01 01:03:51 +07:00
333f617b12 [ChangeLog] Actualize
[ci skip]
2019-03-01 01:02:36 +07:00
ff60ec8f02 [npo] Fix extraction (#20084) 2019-03-01 00:47:18 +07:00
9d9a8676dc [francetv:site] Extend video id regex (closes #20029, closes #20071) 2019-02-28 23:26:52 +07:00
db1c3a9d3f [periscope] Extract width and height (closes #20015) 2019-02-27 03:41:15 +07:00
55b8588f0e [servus] Fix extraction (closes #19297) 2019-02-24 23:20:06 +07:00
f0228f56fb [bbccouk] Make subtitles non fatal (#19651) 2019-02-24 21:01:25 +07:00
8c80603f1a [downloader/external] Add support for rate limit and retries for wget 2019-02-23 01:00:03 +07:00
37b239b3b6 [downloader/external] Fix infinite retries for curl (closes #19303) 2019-02-23 00:43:29 +07:00
caf48f557a [metacafe] Fix family filter bypass (closes #19287) 2019-02-21 05:59:07 +07:00
77a842c892 release 2019.02.18 2019-02-18 02:11:11 +07:00
c76fc5b22a [ChangeLog] Actualize
[ci skip]
2019-02-18 02:10:06 +07:00
388cfbd3d8 [tvp:website] Improve support 2019-02-17 14:27:00 +07:00
d93083789b [tvp:series] Fix extraction 2019-02-17 14:09:30 +07:00
34568dc296 [tvp] Detect unavailable videos 2019-02-17 13:39:00 +07:00
3c9647372e [tvp] Fix description extraction, make thumbnail optional and fix tests 2019-02-17 13:38:21 +07:00
659e93fcf5 [linuxacademy] Add extractor (closes #12207) 2019-02-17 07:12:10 +07:00
c9a0ea6e51 [bilibili] Update keys (closes #19233) 2019-02-17 05:00:16 +07:00
d7d513891b [udemy] Extend _VALID_URLs (closes #14330, closes #15883) 2019-02-17 01:05:01 +07:00
ae65c93a26 [udemy] Update User-Agent and detect captcha (closes #14713, closes #15839, closes #18126) 2019-02-17 00:58:13 +07:00
ba2e3730d1 [noovo] Fix extraction (closes #19230) 2019-02-16 22:45:53 +07:00
2b2da3ba10 [rai] Relax _VALID_URL (closes #19232) 2019-02-15 23:56:29 +07:00
794c1b6e02 [vshare] Pass Referer to download request (closes #19205, closes #19221) 2019-02-14 23:43:16 +07:00
7bee705d8f [openload] Add support for oload.live 2019-02-14 22:28:16 +07:00
6f5c1807f4 [imgur] Use video id as title fallback (closes #18590) 2019-02-13 00:02:29 +07:00
985637cbbf [twitch] Add new source format detection approach (closes #19193) 2019-02-12 00:13:50 +07:00
7d8b89163c [tvplayhome] Fix video id extraction (closes #19190) 2019-02-11 04:41:28 +07:00
d777f3e81c [tvplayhome] Fix episode metadata extraction (closes #19190) 2019-02-11 04:39:23 +07:00
4c0e0dc9dc [rutube:embed] Fix extraction and add support private videos (closes #19163) 2019-02-11 00:49:51 +07:00
f516f44094 [soundcloud] Extract more metadata 2019-02-10 23:44:08 +07:00
e9dee7f1b2 [trunews] Add extractor (closes #19153) 2019-02-09 23:50:27 +07:00
91effe22a0 [linkedin:learning] extract chapter_number and chapter_id(closes #19162) 2019-02-08 07:21:50 +01:00
04eacf5453 release 2019.02.08 2019-02-08 01:12:51 +07:00
f1f5b47255 [ChangeLog] Actualize
[ci skip]
2019-02-08 01:10:12 +07:00
1211bb6dac [YoutubeDL] Improve _make_archive_id (closes #19149) 2019-02-08 01:08:48 +07:00
4de3cb883c [malltv] Fix issues and simplify (closes #17856) 2019-02-08 00:43:31 +07:00
22f5f5c6fc [malltv] Add extractor (closes #18058) 2019-02-08 00:43:26 +07:00
49bd993fd9 [spankbang:playlist] Add extractor (closes #19145) 2019-02-08 00:09:50 +07:00
f06a1cabe8 [spankbang] Extend _VALID_URL 2019-02-08 00:07:29 +07:00
241c5d7d38 [trutv] fix extraction(closes #17336) 2019-02-06 19:38:10 +01:00
8fecc7353d [toutv] fix authentication(closes #16398)(closes #18700) 2019-02-06 13:59:33 +01:00
5dda1edef9 [pornhub] Improve and simplify (closes #19135) 2019-02-05 23:09:24 +07:00
d2d970d07e [pornhub] Fix tags and categories extraction (closes #13720) 2019-02-05 23:08:49 +07:00
48fb963b2f [pornhd] Fix formats extraction 2019-02-05 00:07:37 +07:00
70c3ee1367 [pornhd] Extract like count 2019-02-05 00:06:04 +07:00
07fbfef1c7 [radiocanada] switch to the new media requests(closes #19115) 2019-02-03 12:10:41 +01:00
eecf788b90 [teachable] add support for courses.workitdaily.com (closes #18871) 2019-02-03 09:10:35 +01:00
0efcb5a2fe [vporn] Remove extractor (closes #16276)
Handled by generic extractor
2019-02-03 00:33:45 +07:00
7c5307f4c4 [soundcloud:pagedplaylist] Improve (closes #19086) 2019-02-02 23:40:30 +07:00
6cc6e0c34d [soundcloud:pagedplaylist] Add ie and title to entries (#19022)
rel: https://github.com/rg3/youtube-dl/issues/19022
2019-02-02 23:40:22 +07:00
b9bc1cff72 [drtuber] Extract duration 2019-02-02 06:04:00 +07:00
e9fef7ee4e [YoutubeDL] Fallback to ie_key of matching extractor while making download archive id when no explicit ie_key is provided (#19022) 2019-02-02 05:44:31 +07:00
b6423e6ca2 [soundcloud:user] Update tests 2019-02-02 04:11:32 +07:00
3ef2da2d21 [soundcloud] Fix paged playlists extraction, add support for albums and update client id 2019-02-02 04:00:29 +07:00
49fe4175ae [drtv] Improve preference (closes #19079) 2019-02-01 01:49:33 +07:00
9613e14a92 [openload] Add support for openload.pw and oload.pw (closes #18930) 2019-01-31 00:15:45 +07:00
15e832ff2a [openload] Add support for oload.info 2019-01-30 23:39:02 +07:00
645c4885cf [crackle] authorize media detail request(closes #16931) 2019-01-30 14:44:23 +01:00
7b0f9df23d release 2019.01.30.1 2019-01-30 06:19:36 +07:00
c2a0fe2ea7 [ChangeLog] Actualize
[ci skip]
2019-01-30 06:17:25 +07:00
ce52c7c111 [postprocessor/ffmpeg] Fix avconv processing broken in #19025 (closes #19067) 2019-01-30 06:15:23 +07:00
1063b4c707 release 2019.01.30 2019-01-30 00:08:39 +07:00
ca01e5f903 [ChangeLog] Actualize
[ci skip]
2019-01-30 00:05:32 +07:00
5496754ae4 [fox] Remove unused imports 2019-01-30 00:03:19 +07:00
9868f1ab18 [yourporn] Improve (closes #19061) 2019-01-30 00:01:17 +07:00
41cff90c41 [yourporn] Fix extraction and extract duration (closes #18815, closes #18852)
change cdn to cdn4 for the video_url
2019-01-29 23:58:44 +07:00
a2d821d711 [drtv] Improve extraction (closes #19039)
+ Add support for EncryptedUri videos
+ Extract more metadata
* Fix subtitles extraction
2019-01-29 23:33:09 +07:00
6df196f32e [fox] add support for locked videos using cookies(closes #19060) 2019-01-29 00:31:49 +01:00
41c2c254d3 [fox] fix extraction for free videos(#19060) 2019-01-28 22:39:19 +01:00
a81daba231 [zattoo] Add support for tv.salt.ch 2019-01-29 02:20:46 +07:00
61ff92e11e [postprocessor/ffmpeg] Wrap loglevel args in encodeArgument 2019-01-29 01:59:56 +07:00
1397a790ff [postprocessor/ffmpeg] Disable "Last message repeated" messages which cause non-zero exit status (#19025) 2019-01-28 22:59:00 +07:00
7f903dd8bf [postprocessor/ffmpeg] Do not copy Apple TV chapter tracks while embedding subtitles (closes #19042)
Related issue: https://trac.ffmpeg.org/ticket/6016
2019-01-28 22:57:14 +07:00
2b3afe6b0f [postprocessor/ffmpeg] Disable "Last message repeated" messages which cause non-zero exit status 2019-01-27 22:24:37 -05:00
e71be6ee9f release 2019.01.27 2019-01-27 21:28:09 +07:00
bf8ebc9cfe [ChangeLog] Actualize
[ci skip]
2019-01-27 21:26:28 +07:00
1fcc91663b [vice] fix extraction for locked videos(closes #16248) 2019-01-27 10:53:38 +01:00
30cd1a5f39 [wakanim] detect DRM protected videos 2019-01-26 22:53:09 +01:00
458fd30f56 [extractor/common] Extract season in _json_ld 2019-01-27 04:36:58 +07:00
845333acf6 [wakanim] Add extractor (closes #14374) 2019-01-27 04:14:54 +07:00
252abb1e8b [README.md] Mention more convenience extraction functions 2019-01-26 15:29:19 +07:00
ae18d58297 [usatoday] fix extraction for videos with custom brightcove partner id(closes #18990) 2019-01-25 11:02:15 +01:00
1602a240a7 [drtv] Fix extraction (closes #18989) 2019-01-25 04:16:49 +07:00
0eba178fce [nhk] Extend _VALID_URL (closes #18968) 2019-01-25 04:04:58 +07:00
eb35b163ad [postprocessor/ffmpeg] fallback to ffmpeg/avconv for audio codec detection(closes #681) 2019-01-24 20:23:04 +01:00
118afcf52f [go] fix adobe pass requests for Disney Now(closes #18901) 2019-01-23 22:16:52 +01:00
9713d1d1e0 [openload] Add support for oload.club (closes #18969) 2019-01-24 02:30:12 +07:00
a1e171233d release 2019.01.24 2019-01-24 01:46:23 +07:00
7d311586ed [ChangeLog] Actualize
[ci skip]
2019-01-24 01:44:09 +07:00
e118a8794f [YoutubeDL] Fix typo in string negation implementation and add more tests (closes #18961) 2019-01-24 01:39:39 +07:00
435e382423 release 2019.01.23 2019-01-23 04:46:55 +07:00
0670bdd8f2 [ChangeLog] Actualize
[ci skip]
2019-01-23 04:43:55 +07:00
71a1f61700 [pornhub] Apply scrape detection bypass for all extractors 2019-01-23 04:12:06 +07:00
6510a3aa97 [crunchyroll] Extend _VALID_URL (closes #18955) 2019-01-23 03:55:41 +07:00
278d061a0c [pornhub] Bypass scrape detection (closes #5930) 2019-01-23 03:51:29 +07:00
503b604a31 [vrv] fix oauth signing for python 2(#14307) 2019-01-22 18:21:37 +01:00
4b85f0f9db [vrv] add support for authentication(closes #14307) 2019-01-22 14:38:40 +01:00
19d6991312 [videomore] Improve extraction and fix season extractor (closes #18908) 2019-01-22 03:03:53 +07:00
07f9febc4b [tnaflix] Pass Referer in metadata request (closes #18925) 2019-01-20 22:07:01 +07:00
fad4ceb534 [utils] Fix urljoin for paths with non-http(s) schemes 2019-01-20 20:22:19 +07:00
6945b9e78f [extractor/common] improve jwplayer relative url handling(closes #18892) 2019-01-20 13:31:52 +01:00
29cfcb43da [radiocanada] Relax DRM check 2019-01-20 18:33:09 +07:00
a1a4607598 [vimeo] Fix video password verification for videos protected by Referer HTTP header 2019-01-20 18:21:31 +07:00
73c19aaa9f [hketv] Improve and simplify (closes #18696) 2019-01-20 17:43:11 +07:00
289ef490f7 [hketv] Add extractor 2019-01-20 17:43:11 +07:00
6ca3fa898c [streamango] Add support for fruithosts.net 2019-01-20 16:24:21 +07:00
31fbedc06a [instagram] Add base extractor for playlists and tag extractor 2019-01-20 16:10:46 +07:00
15870747f0 [odnoklassniki] Detect paid videos 2019-01-20 15:15:01 +07:00
fc746c3fdd [test/test_InfoExtractor] add test for #18923 2019-01-20 09:05:12 +01:00
4e58d9fabb [README.md] Fix formatting 2019-01-20 14:23:35 +07:00
2cc779f497 [YoutubeDL] Add negation support for string comparisons in format selection expressions (closes #18600, closes #18805) 2019-01-20 13:48:49 +07:00
379306ef55 [extractor/common] fix typo 2019-01-19 21:35:02 +01:00
f28363ad1f [ted] correct acodec for http formats(#18923) 2019-01-19 21:25:53 +01:00
2bfc1d9d68 [extractor/common] imporove HLS video only format detection(closes #18923) 2019-01-19 21:25:15 +01:00
e2dd132f05 [cartoonnetwork] fix extraction(closes #15664)(closes #17224) 2019-01-17 23:56:37 +01:00
79fec976b0 [vimeo] fix extraction for password protected player URLs(closes #18889) 2019-01-17 09:44:29 +01:00
29639b363d release 2019.01.17 2019-01-17 10:27:17 +07:00
f53cecd796 [ChangeLog] Actualize
[ci skip]
2019-01-17 10:25:50 +07:00
fa4ac365f6 [youtube] Extend JS player signature function name regexes (closes #18890, closes #18891, closes #18893) 2019-01-17 10:24:44 +07:00
bfc8eeea57 release 2019.01.16 2019-01-16 02:24:08 +07:00
b0d73a7456 [ChangeLog] Actualize
[ci skip]
2019-01-16 02:20:10 +07:00
4fe54c128a [youtube] Update tests and add a tests for #18804 2019-01-16 02:18:27 +07:00
a16c7c033a [test/helper] Add support for maxcount and count collection len test checkers 2019-01-16 02:17:49 +07:00
2f483bc1c3 [youtube] Skip unsupported adaptive stream type (#18804) 2019-01-16 01:28:50 +07:00
561b456e2d [youtube] Extract DASH formats from player response (closes #18804) 2019-01-16 01:12:58 +07:00
929ba3997b [funimation] fix extraction(closes #14089) 2019-01-15 10:25:31 +01:00
10026329c2 [skylinewebcams] Fix extraction (closes #18853) 2019-01-14 23:23:51 +07:00
3b983ee471 [curiositystream] add support for non app urls 2019-01-13 15:46:54 +01:00
f1ab3b7de7 [downloader/hls] fix uplynk ad skipping(closes #18824) 2019-01-13 10:03:27 +01:00
d65f6e734b [bitchute] Check formats (#18833) 2019-01-13 03:59:38 +07:00
ed8db0a25c [wistia] Extend _VALID_URL (closes #18823) 2019-01-12 04:58:01 +07:00
60a899bb7e [README.md] Fix typo 2019-01-12 02:15:48 +07:00
cbdc688c41 [postprocessor/ffmpeg] Relax ubuntu ffmpeg version regex 2019-01-12 00:30:06 +07:00
5caa531a1a [postprocessor/ffmpeg] PEP 8 2019-01-11 23:47:23 +07:00
a64646e417 [postprocessor/ffmpeg] sanitize ffmpeg version for Ubuntu and Arch Linux systems(closes #18813) 2019-01-11 15:12:10 +01:00
c469e8808c [playplustv] add support for playplus.com(#18789) 2019-01-10 18:49:13 +01:00
b64f6e690f release 2019.01.10 2019-01-10 23:26:54 +07:00
a4491dd55c [ChangeLog] Actualize
[ci skip]
2019-01-10 23:23:19 +07:00
c3e543893b [youtube] Extract live HLS URL from player response (closes #18799) 2019-01-10 22:46:53 +07:00
432aba1c5e [outsidetv] Add new extractor(closes #18774) 2019-01-10 10:54:46 +01:00
7c072f00d6 [jwplatform] use JW Platform Delivery API V2 and add support for more urls 2019-01-10 10:50:18 +01:00
96c186e1fd [fox] add support National Geographic(closes #17985)(closes #15333)(closes #14698) 2019-01-10 09:05:00 +01:00
4ad159c7b0 [playplustv] Add new extractor(closes #18789) 2019-01-09 20:39:48 +01:00
65615be368 [globo] set GLBID cookie manually(closes #17346) 2019-01-09 19:17:58 +01:00
3c1089dba4 [gaia] Add new extractor(#14605) 2019-01-09 14:23:26 +01:00
6089ff40e7 [youporn] Fix title and description extraction (closes #18748) 2019-01-09 00:37:01 +07:00
2543938bbe [hungama] Add support for videos (closes #17402) 2019-01-08 10:03:44 +07:00
440863ade1 [extractor/common] Use episode name as title in _json_ld 2019-01-08 10:02:49 +07:00
391256dc0e [extractor/common] Add support for movies in _json_ld 2019-01-08 10:02:00 +07:00
06b4b90c70 [hungama] Fix code and extract more metadata (closes #18771) 2019-01-08 09:14:47 +07:00
8cb5c2181a [hungama] Add extractor 2019-01-08 09:14:22 +07:00
0266854f63 [dtube] Update test 2019-01-08 08:46:34 +07:00
bcc334a3c6 [dtube] Fix extraction (closes #18741) 2019-01-08 08:44:42 +07:00
e9a50fba86 [utils] Fix typo 2019-01-07 01:02:34 +07:00
04fb6928da [postprocessor/ffmpeg] Embed subtitles with non-standard language codes (refs #18765) 2019-01-07 00:57:55 +07:00
b7acc83550 [utils] Add language codes replaced in 1989 revision of ISO 639 to ISO639Utils (closes #18765) 2019-01-07 00:57:48 +07:00
de0359c0af [tvnow] Fix and rework extractors, prepare for a switch to the new API (closes #17245, closes #18499) 2019-01-05 03:44:19 +07:00
c87f65e43d [carambatv:page] Fix extraction (closes #18739) 2019-01-04 22:21:53 +07:00
d7c3af7a72 release 2019.01.02 2019-01-02 23:52:54 +07:00
aeb72b3a41 [ChangeLog] Actualize
[ci skip]
2019-01-02 23:51:23 +07:00
2122d7151d [discovery] Use geo verification headers 2019-01-02 23:46:07 +07:00
751e051557 [packtpub] Add support for subscription.packtpub.com (closes #18718) 2019-01-02 17:26:15 +07:00
d226c560a6 Refactor code to use url_result 2019-01-01 23:56:05 +07:00
8437f5089f [README.md] Add long lines policy to coding conventions 2019-01-01 23:50:02 +07:00
1d803085d7 [yourporn] Fix extraction 2019-01-01 23:26:59 +07:00
696f4e4114 [README.md] Add more guide lines for regular expressions 2019-01-01 23:13:39 +07:00
0e713dbb11 [acast:channel] Add support for play.acast.com (closes #18587) 2019-01-01 22:48:06 +07:00
9b5c8751ee [extractors] Add missing age limits 2019-01-01 21:39:18 +07:00
d9f1123c08 [rmcdecouverte] Improve, bypass geo restriction and add support for live (closes #18697) 2019-01-01 20:51:19 +07:00
3d8eb6beb9 [rmcdecouverte] Update _VALID_URL (closes #18595) 2019-01-01 20:50:57 +07:00
38d15ba7f9 [manyvids] Fix extraction (closes #18604, closes #18614) 2019-01-01 20:31:48 +07:00
6b688b8942 [bitchute] Fix extraction (closes #18567) 2019-01-01 18:12:44 +07:00
9d9daed464 release 2018.12.31 2018-12-31 23:59:52 +07:00
32ac3d49ae [ChangeLog] Actualize
[ci skip]
2018-12-31 23:57:46 +07:00
373941c5f0 [bbc] Add support for another embed pattern (closes #18643) 2018-12-31 23:20:40 +07:00
4e1ddc8da9 [npo:live] Add support for npostart.nl (closes #18644) 2018-12-31 21:05:07 +07:00
e4d51e751e [beeg] Fix extraction (closes #18610, closes #18626) 2018-12-31 20:59:53 +07:00
c2dd2dc086 [youtube] Unescape HTML for series (closes #18641) 2018-12-31 19:57:01 +07:00
140a13f5de [youtube] extract more format metadata 2018-12-26 10:56:10 +01:00
825cd268a3 [youtube] detect DRM protected videos(#1774) 2018-12-26 10:56:10 +01:00
63529e935c [youtube] Relax html5 player regexes (closes #18465, closes #18466) 2018-12-23 16:57:10 +07:00
4273caf5c7 [youtube] Extend html5 player regex (closes #17516) 2018-12-23 16:40:16 +07:00
e1a0628797 [liveleak] add support for another embed type and restore original format extraction 2018-12-20 23:22:51 +01:00
835e45abab [crackle] extract ism and http formats 2018-12-19 22:08:01 +01:00
904bb599be [README.md] Add flake8 instructions 2018-12-20 03:22:10 +07:00
65e29cdac3 [twitter] pass referer with card request(closes #18579) 2018-12-18 22:49:01 +01:00
4ee1845454 [mediasite] Extend _VALID_URL even more 2018-12-18 01:55:13 +07:00
cfd13c4c45 [mediasite] Relax _VALID_URL 2018-12-18 00:03:00 +07:00
386d1fea79 [lecturio] Add support for lecturio.de (closes #18562) 2018-12-17 23:43:12 +07:00
7216e9bff7 [discovery] Add support for Scripps Networks watch domains(closes #17947) 2018-12-17 16:35:29 +01:00
4cee62ade0 release 2018.12.17 2018-12-17 05:37:50 +07:00
cbb3e4b14f [ChangeLog] Actualize
[ci skip]
2018-12-17 05:34:55 +07:00
752582183a [ard:beta] Improve extraction robustness, fix subtitles extraction, improve geo restricted videos extraction 2018-12-17 05:29:59 +07:00
1c82122741 [ard:beta] Relax _VALID_URL (closes #18441) 2018-12-17 04:51:57 +07:00
50a498a68e [acast] Extend _VALID_URL 2018-12-17 04:32:59 +07:00
252e172dea [acast] Add support for embed.acast.com 2018-12-17 04:29:12 +07:00
90046d7761 [iprima] Relax _VALID_URL (closes #18515) 2018-12-17 04:10:36 +07:00
c8b3751086 [vrv] fix initial state extraction 2018-12-16 14:29:18 +01:00
21c340b83f [youtube] Fix mark watched (closes #18546) 2018-12-16 19:40:49 +07:00
c984196cf1 [README.md] Bind info dict URLs to a fixed blob (closes #18492) 2018-12-15 23:59:17 +07:00
7f41a598b3 [safari] Add support for learning.oreilly.com (closes #18510) 2018-12-15 23:08:14 +07:00
8fe104947d [youtube] Fix multifeed extraction (closes #18531) 2018-12-15 22:25:12 +07:00
0a05cfabb6 [lecturio] Improve subtitles extraction (closes #18488) 2018-12-11 23:45:02 +07:00
13e17cd28e [uol] fix format url extraction(closes 18480) 2018-12-10 15:02:54 +01:00
102a4e54c5 [teachable] Remove debug output 2018-12-10 10:10:28 +07:00
6e29458f24 [test/testdata/cookies/session_cookies.txt] Fix empty expires test data 2018-12-10 04:30:00 +07:00
59c3940165 [ard:mediathek] Add support for classic.ardmediathek.de (closes #18473) 2018-12-10 01:37:10 +07:00
cefe42c412 release 2018.12.09 2018-12-09 23:11:32 +07:00
24cc64254c [ChangeLog] Actualize
[ci skip]
2018-12-09 23:08:16 +07:00
9e02c2c704 [YoutubeDLCookieJar] Add test for keeping session cookies 2018-12-09 22:57:00 +07:00
5ee7ae5c75 [teachable] Add support for teachable based platform sites (closes #5451, closes #18150, closes #18272) 2018-12-09 22:36:36 +07:00
3ad6dabd33 [aenetworks] add support for History Vault(closes #18460) 2018-12-09 10:04:00 +01:00
5f47a60c5d [imgur] improve gallery and album detection and extraction(closes #9133)(closes #16577)(closes #17223)(closes #18404) 2018-12-09 09:35:17 +01:00
1bab343704 [YoutubeDL] Introduce YoutubeDLCookieJar and clarify the rationale behind session cookies (closes #12929) 2018-12-09 06:47:49 +07:00
1d88b3e6e6 [YoutubeDL] Recognize expires=0 as session cookies and send session cookies with requests 2018-12-09 06:05:37 +07:00
9235b5091c [iprima] Relax _VALID_URL (closes #18453) 2018-12-09 00:01:11 +07:00
c3c098dcf2 [hotstar] fix video data extraction(closes #18386) 2018-12-07 18:52:01 +01:00
8c5879715f [ard:mediathek] Fix title and description extraction (closes #18349) 2018-12-07 03:41:02 +07:00
ebb0449049 [xvideos] Switch to HTTPS (closes #18422) 2018-12-07 03:36:08 +07:00
dfe0a3a9d2 [lecturio] Add extractor (closes #18405) 2018-12-07 03:27:11 +07:00
c976873c5b [nrktv:series] Add support for extra materials 2018-12-07 00:54:58 +07:00
15699ec8b0 [nrktv:season,series] Fix extraction and update tests (closes #17159, closes #17258) 2018-12-07 00:49:24 +07:00
33cc1ea586 [nrktv] Relax _VALID_URL (closes #18304, closes #18387) 2018-12-07 00:00:06 +07:00
ae9d77dab5 [yourporn] Fix extraction (closes #18424) 2018-12-06 23:24:35 +07:00
8bb0c9cc16 [tbs] fix info extraction(fixes #18403) 2018-12-05 07:03:00 +01:00
5547014ad9 [gamespot] add support reviews URLs 2018-12-02 20:01:56 +01:00
ab896fa894 release 2018.12.03 2018-12-03 00:10:20 +07:00
1fa59a928e [ChangeLog] Actualize
[ci skip]
2018-12-03 00:06:54 +07:00
ce18a19be9 [tiktok] Improve extraction and add support for user pages (closes #18135) 2018-12-02 02:42:56 +07:00
1ead840d2c [tiktok] Add extractor (closes #18108) 2018-12-02 02:42:56 +07:00
aa374bc78e [utils] Fix random_birthday to generate existing dates only 2018-12-02 00:05:15 +07:00
3430ff9b07 [pornhub] Use actual URL host for requests (closes #18359) 2018-12-01 16:45:51 +07:00
f012823082 [lynda] Fix authentication (closes #18158) 2018-11-30 01:20:27 +07:00
16597c2f94 [gfycat] Update API endpoint (closes #18333) 2018-11-30 01:07:07 +07:00
adbbdefc81 [hotstar] Add support for alternative app state layout (closes #18320) 2018-11-30 00:48:15 +07:00
053e5b12b2 [azmedien] Fix extraction (closes #18334) 2018-11-30 00:12:18 +07:00
d9df8f120b [vimeo] extract VHX subtitles 2018-11-28 20:13:36 +01:00
ca01d17884 [vimeo] Add support for VHX(Vimeo OTT)(#14835) 2018-11-28 19:53:45 +01:00
d19600df07 [joj] Fix extraction (closes #18280) 2018-11-24 22:14:27 +07:00
641e86e3cf [wistia] Add support for fast.wistia.com (closes #18287) 2018-11-24 21:47:41 +07:00
6864855eb1 [tests] Fix invalid escape sequences 2018-11-23 00:43:42 +07:00
d861a9d581 release 2018.11.23 2018-11-23 00:16:45 +07:00
66173211c4 [ChangeLog] Actualize
[ci skip]
2018-11-23 00:14:43 +07:00
6f2883a2df [mixcloud] base64 decode before decryption 2018-11-21 23:25:38 +01:00
560020da30 [mixcloud] fallback to hardcoded decryption key(closes #18016) 2018-11-21 23:21:05 +01:00
305ce767d5 [travis] Add python 3.8-dev build 2018-11-22 02:34:35 +07:00
157eef3e63 [setup.py] Add python 3.8 classifier 2018-11-22 02:08:41 +07:00
bd2d553c7b [travis] Add python 3.7 build 2018-11-22 02:01:39 +07:00
af60e81e3c [setup.py] Add more relevant classifiers 2018-11-22 02:01:39 +07:00
a843464a7e [nbc] fix NBCNews article extraction(closes #16194) 2018-11-21 12:10:06 +01:00
6866f24494 [foxsports] update test 2018-11-21 12:08:46 +01:00
4e33e0792a [loc] update test 2018-11-21 12:00:50 +01:00
35328915b5 [foxsports] fix extraction(closes #17543) 2018-11-21 09:46:36 +01:00
6c882aa899 [loc] relax _VALID_URL regex and improve formats extraction 2018-11-21 09:46:36 +01:00
183417a50f [ciscolive:search] Add support for pagination 2018-11-21 06:10:43 +07:00
6a6d7f0641 [ciscolive] Fix issues and improve extraction (closes #17984) 2018-11-21 06:10:39 +07:00
05bd5e9c77 [ciscolive] Add extractor 2018-11-21 06:10:30 +07:00
15ed5a2784 [nzz] Relax kaltura regex 2018-11-21 02:50:40 +07:00
2e1280ed43 [sixplay] fix format extraction 2018-11-19 18:15:51 +01:00
8578ea4dcb [bitchute] use _html_search_regex for title extraction 2018-11-18 16:15:27 +01:00
9b27a78a88 [kaltura] limit requested MediaEntry fields 2018-11-18 16:15:27 +01:00
964b989dc8 [americastestkitchen] Add support for zype embeds (closes #18225) 2018-11-18 20:45:25 +07:00
f97c099131 [pornhub] Move test to correct place 2018-11-18 11:14:46 +07:00
1febf99da1 [pornhub] Add pornhub.net alias 2018-11-18 06:26:08 +07:00
4167148fa4 [nova:embed] Fix extraction (closes #18222) 2018-11-18 01:11:10 +07:00
5bb0479269 release 2018.11.18 2018-11-18 00:11:54 +07:00
02df855e13 [ChangeLog] Actualize
[ci skip]
2018-11-18 00:07:40 +07:00
006374e3ae [wwe] Fix issues, extract subtitles and add support for playlists (closes #14781, closes #17450) 2018-11-17 23:59:52 +07:00
11d19ff503 [wwe] Add extractor 2018-11-17 23:59:46 +07:00
a640c4d226 [vk] Detect geo restriction 2018-11-17 20:59:13 +07:00
d0058c76d5 [openload] Use original host during extraction (closes #18211) 2018-11-17 16:59:20 +07:00
0919cd4d01 [atvat] Fix extraction (closes #18041) 2018-11-17 00:18:50 +07:00
2599956c9f [rte] Add support for new API endpoint (closes #18206) 2018-11-17 00:07:59 +07:00
9b9b3501c5 [tnaflixnetwork:embed] Fix extraction (closes #18205) 2018-11-16 22:55:35 +07:00
730c0d12a0 [picarto] Extract more metadata (closes #16518) 2018-11-11 16:17:59 +07:00
f17a24a6df [picarto] Use API and add token support
This is just more reliable than trying to extract
it from the page itself.
2018-11-11 16:17:56 +07:00
83852e57bf [zype] Add extractor (closes #18143) 2018-11-11 00:44:49 +07:00
96a91b1551 [vivo] Fix extraction (closes #18139) 2018-11-10 23:37:27 +07:00
cab26223bf [ruutu] Update API endpoint (closes #18138) 2018-11-10 15:26:04 +07:00
532782ade1 release 2018.11.07 2018-11-07 01:38:25 +07:00
f81d44aab6 [ChangeLog] Actualize
[ci skip]
2018-11-07 09:58:08 +07:00
2511eee215 [youtube] Add another JS signature function name regex (closes #18091, closes #18093, closes #18094) 2018-11-07 09:55:59 +07:00
0df514f07e [facebook] fix tahoe request(closes #17171) 2018-11-06 21:22:27 +01:00
432cd48410 [cliphinter] Fix extraction (closes #18083) 2018-11-06 23:29:42 +07:00
c0345b825f [youtube:playlist] Add support for invidio.us (closes #18077) 2018-11-05 19:08:39 +07:00
2004e2210b [osnateltv] Update host 2018-11-05 17:09:57 +07:00
16d896b2a7 [zattoo] Arrange API hosts for derived extractors (closes #18035) 2018-11-05 15:52:46 +07:00
22e07ce502 [README.md] Improve documentation on safe metadata extraction and add more examples 2018-11-05 00:11:36 +07:00
dbdaaa231a [youtube] Add fallback metadata extraction from videoDetails (closes #18052) 2018-11-03 06:26:16 +07:00
38c32dbf19 release 2018.11.03 2018-11-03 02:57:48 +07:00
a085410936 [ChangeLog] Actualize
[ci skip]
2018-11-03 02:56:14 +07:00
6895ea4d3f [laola1tv:embed] Set correct stream access URL scheme (closes #16341) 2018-11-03 02:45:37 +07:00
faac1c1f70 [ehftv] Add extractor (closes #15408) 2018-11-03 02:45:32 +07:00
573531dcfb [azmedien] Simplify (closes #17746) 2018-11-03 01:33:42 +07:00
da56fb631f [azmedien] Adopt to major site redesign (closes #17745) 2018-11-03 01:33:36 +07:00
95e42d7336 [extractor/common] Ensure response handle is not prematurely closed before it can be read if it matches expected_status (resolves #17195, closes #17846, resolves #17447) 2018-11-03 01:18:20 +07:00
cf0db4d997 [twitcasting] Improve extraction and fix issues (closes #17981) 2018-11-03 00:28:25 +07:00
036f905161 [twitcasting] Add extractor 2018-11-03 00:28:21 +07:00
4b6aca17cc [orf:tvthek] Improve extraction and remove unused code (closes #17956, closes #18024) 2018-11-02 23:47:23 +07:00
c620694c97 [orf:tvthek] Fix extraction (closes #17737)
use _extract_m3u8_formats and _extract_f4m_formats helper functions
closes #17737
2018-11-02 23:47:17 +07:00
061ea3a776 [openload] Add support for oload.fun 2018-11-02 23:08:41 +07:00
c70ba664f1 [njpwworld] Fix authentication (closes #17427) 2018-11-01 01:40:02 +07:00
f16679e843 [cnbc:video] Fix _VALID_URL (#17110) 2018-10-30 04:57:51 +07:00
b14475724b [linkedin:learning:course] use url_transparent type for playlist entries 2018-10-29 21:49:12 +01:00
aa7e974a2a [linkedin:learning] Add new extractor(closes #13545) 2018-10-29 19:28:29 +01:00
9aac22c195 [theplatform] Improve error detection (#13222) 2018-10-30 00:22:44 +07:00
94db1f7f3b [cnbc] Simplify extraction (closes #14280, closes #17110) 2018-10-29 23:55:55 +07:00
ffa7b2bfee [cbnc] Add support for new URL schema (closes #14193) 2018-10-29 23:54:14 +07:00
2943397e87 [aparat] Improve extraction and extract more metadata (closes #17445, closes #18008) 2018-10-29 23:32:45 +07:00
9c4a83a1be [aparat] Fix extraction 2018-10-29 23:31:24 +07:00
9ff558f67f release 2018.10.29 2018-10-29 00:39:29 +07:00
c2fe21efaa [ChangeLog] Actualize
[ci skip]
2018-10-29 00:38:06 +07:00
476cf548e1 [sportbox] Improve extraction, add support for matchtv.ru and fix video id (closes #17978) 2018-10-29 00:21:50 +07:00
bebef10909 [extractor/common] Add validation for JSON-LD URLs 2018-10-29 00:21:45 +07:00
4c237ab787 [sportbox] Fix extraction 2018-10-29 00:21:40 +07:00
a1d1c63678 [screencast] Improve extraction (closes #14617, closes #17990) 2018-10-28 23:26:30 +07:00
1fafb32984 [screencast] Fix extraction (closes #14590) 2018-10-28 23:26:30 +07:00
c901cc38e5 [openload] Add support for oload.icu 2018-10-28 22:51:29 +07:00
022218f2f0 [ivi] Add support for ivi.tv 2018-10-28 22:49:10 +07:00
08c7d3dade [crunchyroll] Improve extraction failsafeness (closes #17991) 2018-10-28 22:12:54 +07:00
5e733b066a [dailymail] fix format extraction(closes #17976) 2018-10-26 05:41:57 +01:00
7d9e858132 [viewster] reduce format requests 2018-10-26 05:40:49 +01:00
b99b0bcfa0 [cwtv] handle api errors(closes #17905) 2018-10-17 06:22:56 +01:00
baeabf7742 [rutube] Use geo verification headers (closes #17897) 2018-10-16 23:19:44 +07:00
582797d780 [brightcove] remove unused variable 2018-10-15 20:47:12 +01:00
160c2773f6 [brightcove:legacy] add another fall back to brightcove:new 2018-10-15 18:41:57 +01:00
ee5fe42e44 [brightcove:legacy] fall back to brightcove:new(#13912) 2018-10-15 17:54:38 +01:00
f0ee386851 [tv3] remove extractor(closes #10461)(closes #15339) 2018-10-15 16:26:29 +01:00
a94e7c195e [ted] fix extraction for http and rtmp formats(closes #5941)(closes #17572)(closes #17894) 2018-10-15 11:52:01 +01:00
5d90a8a5f3 [openload] Add support for oload.cc 2018-10-07 20:05:45 +07:00
19a352854f [patreon] extract post_file url(#17792) 2018-10-05 22:45:04 +01:00
c9d891f19a [patreon] fix extraction(closes #14502)(closes #10471) 2018-10-05 20:11:01 +01:00
d96f976b0c release 2018.10.05 2018-10-05 02:31:30 +07:00
2e7ed29e34 [ChangeLog] Actualize
[ci skip]
2018-10-05 02:29:52 +07:00
21c1a00dd7 [pluralsight] Improve authentication (closes #17762) 2018-10-05 02:27:14 +07:00
0082f44a08 [dailymotion] Improve metadata extraction (closes #17706) 2018-10-05 02:03:25 +07:00
f60b9803a4 [dailymotion] Fix extraction (closes #17699) 2018-10-05 02:03:16 +07:00
d98cb62e55 [crunchyroll] switch to HTTPS for RpcApi(closes #17749) 2018-10-02 19:43:06 +01:00
05e7c184da [hotstar] fix extraction in python 2(closes #17696) 2018-10-02 06:07:43 +01:00
66d106f270 [philharmoniedeparis] Fix extraction and add support for pad.philharmoniedeparis.fr (closes #17705) 2018-10-01 23:29:49 +07:00
3c7da54c92 [jamendo] Add support for licensing.jamendo.com (closes #17724) 2018-10-01 22:05:18 +07:00
9795d93316 [openload] Add support for oload.cloud (closes #17710) 2018-10-01 21:48:59 +07:00
365343131d [pluralsight] Fix subtitles extraction (closes #17726, closes #17728) 2018-10-01 21:45:24 +07:00
85fa80d5f9 [vimeo] Add another config regex (closes #17690) 2018-10-01 21:13:43 +07:00
245cbb33bc [spike] fix Paramount Network extraction(closes #17677) 2018-09-28 15:13:25 +01:00
85cd69adcb [hotstar] fix extraction(closes #14694)(closes #14931)(closes #17637) 2018-09-26 08:13:36 +01:00
4c89a675dd release 2018.09.26 2018-09-26 11:58:25 +07:00
3d3499742c [ChangeLog] Actualize
[ci skip]
2018-09-26 11:56:15 +07:00
c17e100b96 [pluralsight] Fix subtitles extraction (closes #17671) 2018-09-26 09:27:40 +07:00
8fd12a0831 [mediaset] Improve embed support (closes #17668) 2018-09-26 05:38:41 +07:00
60ce0c67fd [README.md] Document channel meta fields for output template 2018-09-25 23:43:41 +07:00
cd5a74a28e [youtube] Add support for invidio.us (closes #17613) 2018-09-24 00:14:49 +07:00
f6d7f7b474 [zattoo] Add support for more zattoo platform sites 2018-09-23 23:35:37 +07:00
21160a1792 [zattoo] Fix extraction (closes #17175) 2018-09-23 21:34:47 +07:00
4ac73fc170 [popcorntv] Remove debug output 2018-09-19 22:16:43 +07:00
28fcb7b061 [raiplay:playlist] Remove a debug leftover print() 2018-09-19 09:48:39 +07:00
3a9c928426 release 2018.09.18 2018-09-18 01:46:36 +07:00
d9b1cec171 [ChangeLog] Actualize
[ci skip]
2018-09-18 01:44:55 +07:00
e504b09070 [adobepass] Don't pollute default headers dict 2018-09-17 22:15:27 +07:00
c8f6ab8c38 [udemy] Don't pollute default headers dict 2018-09-17 22:14:53 +07:00
e2f61598be [twitch] Don't pollute default headers dict 2018-09-17 22:14:28 +07:00
c11485162b [youtube] Don't pollute default query dict (closes #17593) 2018-09-17 22:13:39 +07:00
1084563eaa [crunchyroll] Prefer hardsubless formats and formats in locale language 2018-09-16 23:54:58 +07:00
d9b0d118ad [vrv] Make format ids deterministic 2018-09-16 23:54:58 +07:00
8b40c92724 [vimeo] redirect to feature url only in the case of a trailer(closes #14591) 2018-09-15 06:31:15 +01:00
3661ebf2b6 [pornhub] Extract upload date (closes #17574) 2018-09-15 02:04:43 +07:00
0e7b8d3eac [extractor/common] Fix typos 2018-09-15 01:53:01 +07:00
127103b643 [porntube] Extract channel meta fields 2018-09-15 01:25:32 +07:00
d03beddf0f [vimeo] Extract channel meta fields 2018-09-15 01:24:48 +07:00
dd4c449219 [youtube] Extract channel meta fields (closes #9676, closes #12939) 2018-09-15 01:24:26 +07:00
6f1f59f39c [extractor/common] Introduce channel meta fields 2018-09-15 01:23:36 +07:00
15bf2ca0da [porntube] Fix extraction (closes #17541) 2018-09-15 00:50:36 +07:00
0f2aa0dcaa [asiancrush] Fix extraction (closes #15630) 2018-09-14 23:56:03 +07:00
db348e8849 [twitch:clips] Extend _VALID_URL (closes #17559) 2018-09-14 23:26:38 +07:00
f5b0175349 [vzaar] Add support for HLS 2018-09-11 02:41:05 +07:00
79facb2773 [tube8] Fix metadata extraction (closes #17520) 2018-09-11 02:32:01 +07:00
96dbf70de6 [eporner] Extract JSON-LD (closes #17519) 2018-09-11 02:32:01 +07:00
8476b4fd91 release 2018.09.10 2018-09-10 02:48:37 +07:00
14f577e31c [ChangeLog] Actualize
[ci skip]
2018-09-10 02:45:44 +07:00
25d110be30 [utils] Properly recognize AV1 codec (closes #17506) 2018-09-10 02:37:22 +07:00
a2637a2dda [iprima] Add support for prima.iprima.cz (closes #17514) 2018-09-10 01:34:49 +07:00
2e4350eec6 [generic] PEP 8 (#17455) 2018-09-08 17:24:59 +07:00
2c9d3b9962 [seznamzpravy] PEP 8 (#17455) 2018-09-08 17:24:48 +07:00
13ef64fd93 [motherless] PEP 8 (#17455) 2018-09-08 17:24:34 +07:00
6f9f3340bb [dtube] PEP 8 (#17455) 2018-09-08 17:24:09 +07:00
ae2384ff5f [tele5] Improve extraction (closes #7805, closes #7922, closes #17331, closes #17414) 2018-09-08 16:08:48 +07:00
d0de6a287a [tele5] Add extractor 2018-09-08 16:08:44 +07:00
d0c5fabc12 [nbc] Fix extraction of percent encoded URLs (closes #17374) 2018-09-08 14:44:06 +07:00
ad98d2eb74 release 2018.09.08 2018-09-08 03:42:28 +07:00
a41a506077 [ChangeLog] Actualize
[ci skip]
2018-09-08 03:40:06 +07:00
9a47fa35dd [youtube] Fix extraction (closes #17457, closes #17464) 2018-09-08 03:36:10 +07:00
2d4fe594c6 [pornhub:uservideos] Add support for new URLs (closes #17388) 2018-09-06 00:51:20 +07:00
09322cccdb [iprima] Confirm adult check (closes #17437) 2018-09-06 00:22:53 +07:00
aa1d5eb905 [slideslive] make the check for video_service_name case-insensitive(closes #17429) 2018-09-04 10:37:51 +01:00
93284ff2ea [radiojavan] Improve extraction (closes #17151) 2018-09-03 02:53:51 +07:00
0a9a8118ce [radiojavan] Fix extraction 2018-09-03 02:53:41 +07:00
3d08f63dc5 [generic] Skip unsuccessful jwplayer extraction (closes #16735) 2018-09-03 02:15:17 +07:00
27d8e089a2 release 2018.09.01 2018-09-01 18:40:23 +07:00
7bbc1b189a [ChangeLog] Actualize
[ci skip]
2018-09-01 18:36:18 +07:00
0b87e88453 [ard] Add support for one.ard.de 2018-09-01 16:42:30 +07:00
4d59db5b90 [niconico] Fix extraction on python3 (closes #17393) 2018-09-01 16:04:45 +07:00
4627995882 [crunchyroll] limit VRVIE inheritance to CrunchyrollIE 2018-09-01 10:04:10 +01:00
7f2611cb5b [ard] extract f4m formats 2018-09-01 08:40:38 +01:00
54a5be4dba [crunchyroll] parse vilos media data(closes #17343) 2018-09-01 08:16:41 +01:00
ed6919e737 [ard] beta mediathek: make regexp for JSON more robust 2018-09-01 01:59:13 +02:00
2b83da2463 [ard] Better format handling
Skip f4m, doesn't work (yet); correctly extract m3u8, and prefer plain HTTP files.
2018-09-01 00:45:36 +02:00
c1a37eb24a [ard] Add support for Beta ARD Mediathek
Thanks to https://blog.fefe.de/?ts=a577685d for pointing out support is missing.
2018-09-01 00:18:17 +02:00
4991e16c2a [bandcamp] Extract more metadata (closes #13197) 2018-08-31 03:35:55 +07:00
14b7a24c19 [bandcamp] Extract track_number (closes #17266) 2018-08-31 02:32:35 +07:00
73f3bdbeb4 [internazionale] Fix extraction of non-available-abroad videos 2018-08-31 02:15:46 +07:00
9e21e6d96b [utils] Improve remote address skipping and add support for python 2.6 (closes #17362) 2018-08-29 01:18:03 +07:00
8959018a5f [utils] Skip remote IP addresses non matching to source address' IP version (closes #13422) 2018-08-29 01:17:53 +07:00
eebbce5656 release 2018.08.28 2018-08-28 03:10:09 +07:00
56213aff1d [ChangeLog] Actualize
[ci skip]
2018-08-28 03:07:18 +07:00
409b9324da [youtube:playlist] Add support for music album playlists (OLAK5uy_ prefix) (closes #17361) 2018-08-28 02:20:32 +07:00
02df41354c [bitchute] Fix extraction by pass custom User-Agent 2018-08-27 22:04:56 +07:00
dd88fd65a5 [webofstories:playlist] Fix extraction (closes #16914) 2018-08-26 21:42:30 +07:00
287cf7e443 [generic] Remove unused import 2018-08-26 20:09:02 +07:00
dac6f7654a [tvplayhome] Add extractor (closes #17344) 2018-08-26 20:08:55 +07:00
e0b6e98871 [generic] Allow relative src for videojs embeds (closes #17324) 2018-08-24 23:12:53 +07:00
beff09505c [xfileshare] Add support for vidto.se (closes #17317) 2018-08-24 04:00:35 +07:00
135e6a1c10 [vidzi] Add support for vidzi.nu (closes #17316) 2018-08-24 02:36:56 +07:00
c707d2067d [nova:embed] Add extractor (closes #17282) 2018-08-22 23:18:07 +07:00
4c86163b60 release 2018.08.22 2018-08-22 02:32:18 +07:00
b662273989 [ChangeLog] Actualize
[ci skip]
2018-08-22 02:28:25 +07:00
df4d817bc3 [kinopoisk] Add extractor (closes #17283) 2018-08-22 02:19:30 +07:00
db192b2932 [yourporn] Add extractor (closes #17298) 2018-08-22 01:44:22 +07:00
52007de8ca [go] Add support for disneynow.go.com (closes #16299, closes #17264) 2018-08-22 01:14:47 +07:00
28f96cf407 [6play] Add support for play.rtl.hr 2018-08-22 00:06:27 +07:00
eda86b4335 [anvato] Fallback to generic API key for access key to API key lookup (closes #16788, closes #17254) 2018-08-21 23:45:18 +07:00
bf1245d236 [lci] Fix extraction (closes #17274) 2018-08-20 02:15:48 +07:00
6f356cbbcf [bbccouk] Extend _ID_REGEX (closes #17270) 2018-08-20 02:05:07 +07:00
0a74b45191 [cwtv] fix extraction(closes #17256) 2018-08-17 11:59:49 +01:00
d6ef8b4dd4 [nova] Fix extraction (closes #17241) 2018-08-16 00:11:41 +07:00
60c0856223 [utils] Use pure browser header for User-Agent (closes #17236) 2018-08-14 23:27:12 +07:00
57c68ec4c3 [generic] Add support for expressen embeds 2018-08-14 22:51:44 +07:00
24e0cd709f [raywenderlich] Adapt to site redesign (closes #17225) 2018-08-13 00:15:59 +07:00
4779420ce8 [redbulltv] add support redbull.com tv URLs(closes #17218) 2018-08-12 05:31:34 +01:00
de4c41b437 [bitchute] Improve page offset 2018-08-12 01:52:50 +07:00
b65e3b0636 [bitchute] Add extractor (closes #14052) 2018-08-12 01:47:10 +07:00
d37dc6e1c9 [clyp] Add support for token protected media (closes #17184) 2018-08-07 23:27:08 +07:00
a62460aa21 [imdb] Fix extension extraction (closes #17167) 2018-08-06 04:37:03 +07:00
d588d4a5a6 [.gitignore] Add .vscode 2018-08-05 22:10:01 +07:00
81cc22bab6 release 2018.08.04 2018-08-04 01:23:24 +07:00
20f96f64bd [ChangeLog] Actualize
[ci skip]
2018-08-04 01:21:23 +07:00
af322eb830 [funk:channel] Improve byChannelAlias extraction (closes #17142) 2018-08-04 00:26:58 +07:00
cb1c3a3c07 [twitch] Update cliend id and modernize (closes #17126) 2018-08-03 22:44:31 +07:00
48afc6ca3e [twitch] Fix authentication (closes #17024) 2018-08-03 22:43:23 +07:00
644921b372 [twitch:vod] Improve _VALID_URL (closes #17135) 2018-08-02 23:16:15 +07:00
19b9de13c4 [watchbox] Fix extraction (closes #17107) 2018-07-30 23:28:44 +07:00
6f2d82a5a0 [pbs] Fix extraction (closes #17109) 2018-07-30 23:10:40 +07:00
7ff129d3ea [theplatform] Relax _VALID_URL (closes #16181) 2018-07-30 03:15:06 +07:00
9d1b213845 [viqeo] Add extractor (closes #17066) 2018-07-30 03:05:36 +07:00
5484828418 release 2018.07.29 2018-07-29 07:02:18 +07:00
4eecef84f3 [ChangeLog] Actualize
[ci skip]
2018-07-29 06:59:39 +07:00
b2286f8fb2 [crunchyroll:playlist] Restrict _VALID_URL (closes #17069) 2018-07-29 06:56:52 +07:00
4938c8d573 [pornhub] Add support for subtitles (closes #16924) 2018-07-29 06:24:10 +07:00
1a88fc5a69 [ceskatelevize] Use https for API call (refs #16997) 2018-07-29 06:04:59 +07:00
38e87f6c2a [utils] Remove return from __init__ 2018-07-29 05:52:42 +07:00
ec240a4369 [dailymotion:playlist] fix extraction(closes #16894) 2018-07-28 20:30:44 +01:00
cd3a3ff93b [ted] Improve extraction and update tests 2018-07-28 22:09:53 +07:00
9a984265b9 [ted] Fix extraction for videos without nativeDownloads (closes #16756, closes #17085) 2018-07-28 21:26:23 +07:00
a098c99f0d [telecinco] fix extraction(closes #17080) 2018-07-28 06:55:42 +01:00
8e37a7e4cc [mitele] reduce number of requests and update tests 2018-07-28 06:55:42 +01:00
722f1a0f8f [README.md] Actualize Firefox cookie export add-on
Previous one does not work with newer Firefox versions
2018-07-28 00:18:41 +07:00
0c7b4f49eb [rai] return non http relinker URL intact(closes #17055) 2018-07-26 08:11:06 +01:00
ad1bc71a8a [vk] fix extraction for inline only videos(fixes #16923) 2018-07-26 07:25:07 +01:00
b5dec62ca6 [streamcloud] Fix extraction (closes #17054) 2018-07-25 23:07:12 +07:00
631f93ee2d [facebook] fix tahoe request for authenticated users(closes #16655) 2018-07-23 06:20:18 +01:00
d4e7065111 Credit @Kerruba for #16328 2018-07-22 21:36:58 +07:00
234a85858c Credit @tmsbrg for #15462 2018-07-22 21:35:38 +07:00
a789d1cc90 Credit @nathanrossi for #16554 2018-07-22 21:34:34 +07:00
694079dff7 Credit @mrfade for #16269 and #16271 2018-07-22 21:31:46 +07:00
d94fb1225e Credit @dnet for #16174 2018-07-22 21:29:25 +07:00
7930f91494 Credit @haasn for #16326 2018-07-22 21:27:28 +07:00
a702056fbe Credit @bastiandg for #16189 2018-07-22 21:26:12 +07:00
8fd2a7be37 [puhutv] Improve extraction (closes #16269) 2018-07-22 20:36:48 +07:00
6de82b4476 [puhutv] Add extractor (closes #16010) 2018-07-22 20:36:48 +07:00
8e66ffc3b7 release 2018.07.21 2018-07-21 21:00:18 +07:00
6f27998e75 [ChangeLog] Actualize
[ci skip]
2018-07-21 20:58:30 +07:00
3052a30d42 Improve URL extraction 2018-07-21 19:08:28 +07:00
4ecf300d13 [iwara] Improve extraction 2018-07-21 18:03:58 +07:00
af03000ad5 [utils] Introduce url_or_none 2018-07-21 18:03:58 +07:00
b96b4be461 [bbc] add support for BBC Radio Play pages(closes #17022) 2018-07-21 11:50:14 +01:00
edb0e17188 [iwara] Fix download URLs (closes #17026) 2018-07-21 17:41:33 +07:00
e9c671d5e8 [utils] Allow JSONP with empty func name (closes #17028) 2018-07-21 12:30:18 +07:00
fd62b36680 [vrtnu] Relax title extraction and extract JSON-LD (closes #17018) 2018-07-20 02:39:20 +07:00
25586c601c [theplatform] PEP 8
[ci skip]
2018-07-20 00:49:14 +07:00
ecb6b6ae2d [viu] Pass area id 2018-07-20 00:46:50 +07:00
c258570edd [viu] Pass Referer and Origin headers (closes #16992) 2018-07-20 00:01:43 +07:00
6fc09f0155 [vimeo] Add another config regex (closes #17013) 2018-07-19 23:15:27 +07:00
11330f5121 [facebook] Extract view count and update tests (closes #16942) 2018-07-19 02:26:05 +07:00
8da17f9680 [dailymotion] Improve description extraction (closes #16984) 2018-07-19 02:04:05 +07:00
c63f5fb863 [slutload] Fix and improve extraction (closes #17001) 2018-07-19 01:59:00 +07:00
38f1eb0ac3 [mediaset] fix extraction(closes #16977) 2018-07-18 18:34:04 +01:00
371dcc1dd4 [theplatform] add support for theplatform Top-level domain customization(#16977) 2018-07-18 18:34:04 +01:00
bd21ead2a2 [extractor/common] add support for DASH and MSS formats extraction in SMIL manifests 2018-07-18 18:34:04 +01:00
905eef2b06 [imgur] Allow digits in filename extension 2018-07-18 23:47:26 +07:00
79367a9820 [pornhub] Improve extraction and extract all formats (closes #12166, closes #15891, closes #16262, closes #16959) 2018-07-14 18:05:06 +07:00
40a051fa9f release 2018.07.10 2018-07-10 02:09:51 +07:00
7e8e948cf7 [ChangeLog] Actualize
[ci skip]
2018-07-10 02:08:15 +07:00
4b3ee09886 [nrktv] Add support for new season and serie URL schema 2018-07-10 00:27:24 +07:00
79fd7320e2 [nrktv] Add support for new episode URL schema (closes #16909) 2018-07-09 23:44:05 +07:00
0685d9727b [utils] Share JSON-LD regex 2018-07-09 23:43:05 +07:00
e06632e3fe [downloader/dash] Improve error handling (#16927) 2018-07-08 08:22:56 +07:00
69fcdb845b [frontendmasters] Fix issues and improve extraction (closes #3661, closes #16328) 2018-07-08 00:55:49 +07:00
6868d272e5 [frontendmasters] Add extractor 2018-07-08 00:49:15 +07:00
4742150788 [funk] Fix extraction (closes #16918) 2018-07-06 23:50:02 +07:00
4e71dfd819 [README.md] Rename OS X to macOS 2018-07-05 22:17:18 +07:00
1ed0b2f74d [watchbox] Fix extraction (closes #16904) 2018-07-05 02:22:15 +07:00
e15141adae [dplayit] Sort formats 2018-07-05 02:14:50 +07:00
94fef94d9c [dplayit] Fix extraction (closes #16901) 2018-07-05 02:14:06 +07:00
9a6628aaf9 [youtube] Improve login error handling (closes #13822) 2018-07-05 00:37:32 +07:00
689af4960e release 2018.07.04 2018-07-04 04:59:21 +07:00
d5de0f21b9 [ChangeLog] Actualize
[ci skip]
2018-07-04 04:57:17 +07:00
24d26ab380 [lynda] PEP 8 2018-07-04 04:49:03 +07:00
836ef4840f [pluralsight] Switch to graphql (closes #16889, closes #16899) 2018-07-04 04:48:40 +07:00
5621c3222e [lynda] Simplify login and improve error capturing (#16891) 2018-07-03 02:47:09 +07:00
db5debf313 [go90] add support for embed urls(closes #16873) 2018-07-01 22:41:32 +01:00
8cee692b8b [go90] detect geo restriction error and pass geo verification headers(closes #16874) 2018-07-01 22:41:32 +01:00
973b6ceebb [vlive] Fix live streams extraction 2018-07-01 21:19:17 +07:00
eca1f0d115 [extractor/common] Properly escape % in MPD templates (closes #16867) 2018-07-01 02:11:36 +07:00
2160768a21 [npo] Fix typo (closes #16872) 2018-06-30 23:39:56 +07:00
267d81962a [mediaset] Fix issues and extract all formats (closes #16568) 2018-06-30 02:19:02 +07:00
9cf648c92b [mediaset] Add support for new videos 2018-06-30 02:17:51 +07:00
5e8e2fa51f [extractor/common] Use source URL as Referer for HTML5 entries (closes #16849) 2018-06-29 01:25:05 +07:00
d4a24f4091 Prefer ffmpeg over avconv by default (closes #8622) 2018-06-29 01:09:14 +07:00
acbd0ff5df [dctptv] Restore extraction based on REST API (closes #16850) 2018-06-29 00:35:05 +07:00
7b393f9cc5 [svt] Improve extraction and add support for pages (closes #16802) 2018-06-28 04:29:11 +07:00
c3bcd206eb [porncom] Fix extraction (closes #16808) 2018-06-26 00:01:06 +07:00
1f6cc5807e release 2018.06.25 2018-06-25 02:26:02 +07:00
c306f076ec [ChangeLog] Actualize
[ci skip]
2018-06-25 02:17:14 +07:00
a0949fec08 [joj] Relax _VALID_URL (closes #16771) 2018-06-24 23:57:22 +07:00
74caf528bc [brightcove] workaround sonyliv DRM protected videos(closes #16807) 2018-06-24 12:02:32 +01:00
9fb62e35f6 [motherless:group] Fix _VALID_URL 2018-06-21 23:39:13 +07:00
b71cc71910 [motherless] Fix extraction (closes #16786) 2018-06-21 23:38:32 +07:00
a4ec45179e [itv] Sort imports 2018-06-21 23:12:40 +07:00
30374f4d40 [itv] Make SOAP request non fatal and extract metadata from a webpage (closes #16780) 2018-06-21 23:06:58 +07:00
91aa502d91 [foxnews:insider] Remove extractor (#15810)
Now covered by foxnews:article
2018-06-20 23:59:37 +07:00
f51f526b0a [foxnews] Add support for iframe embeds (closes #15810, closes #16711) 2018-06-20 23:53:37 +07:00
c9b983ff82 release 2018.06.19 2018-06-19 23:16:04 +07:00
e730508827 [ChangeLog] Actualize
[ci skip]
2018-06-19 23:12:53 +07:00
8b4b400aef [peertube] Improve generic support (closes #16733) 2018-06-19 23:00:36 +07:00
e12b4b8bcc [6play] use geo verfication headers 2018-06-19 10:35:57 +01:00
18806e3b6b [rtbf] fix extraction for python 3.2 and older 2018-06-18 19:10:43 +01:00
713afa705c [vgtv] Improve HLS formats extraction 2018-06-18 23:15:38 +07:00
721a877d2f [vgtv] Add support for www.aftonbladet.se/tv/ URLs 2018-06-18 23:08:35 +07:00
9283d4ea03 [bbccouk] Use expected_status 2018-06-18 04:54:59 +07:00
00a429bea3 [markiza] Expect 500 status code 2018-06-18 04:54:52 +07:00
d391b7e23d [extractor/common] Introduce expected_status for convenient accept of failed HTTP requests
Useful when some non-success (2xx) HTTP status codes should be considered normal. Previously this required to manually catch corresponding exceptions and read the response.
2018-06-18 04:54:08 +07:00
075a13d3e9 [compat] Introduce compat_integer_types 2018-06-18 04:52:58 +07:00
8ba84e4600 [tvnow] try all clear manifest urls(closes #15361) 2018-06-17 20:41:09 +01:00
858cf4dc29 release 2018.06.18 2018-06-18 01:34:36 +07:00
9e761fe6f5 [ChangeLog] Actualize
[ci skip]
2018-06-18 01:31:49 +07:00
ce0edda0f9 [markiza] Add extractors (closes #16750) 2018-06-18 01:17:47 +07:00
0adf213d8c [wat] try all supported adaptive urls 2018-06-17 15:56:52 +01:00
8b183bd5f8 [tf1] try all supported adaptive urls 2018-06-17 15:53:29 +01:00
1882511754 [6play] add support for rtlplay.be and extract hd usp formats 2018-06-17 12:01:14 +01:00
764cd4e6f3 [rtbf] improve extraction
- add support for audio and live streams(closes #11923)(closes #9638)
- extract HLS, DASH and all HTTP formats
- extract subtitles
- fixup specific http urls(fixes #16101)
2018-06-17 03:13:41 +01:00
734d461ca0 [expressen] Add extractor 2018-06-16 21:15:06 +07:00
81c5df4f2c [vidzi] Fix extraction (closes #16678) 2018-06-16 05:08:44 +07:00
87f89dacdd [pbs] Improve extraction (closes #16623, closes #16684) 2018-06-16 02:55:20 +07:00
9b0b627534 [downloader/rtmp] Fix downloading in verbose mode (closes #16736) 2018-06-15 02:59:15 +07:00
61cb66830f [bilibili] Restrict cid regex (closes #16638, closes #16734) 2018-06-14 22:40:30 +07:00
c797db4a2f release 2018.06.14 2018-06-14 01:24:53 +07:00
03eef0f032 [ChangeLog] Actualize
[ci skip]
2018-06-14 01:22:42 +07:00
aa56061627 [discoverynetworks] Add support for disco-api videos(closes #16724) 2018-06-13 16:46:59 +01:00
18d66f0410 [dailymotion] use compat_struct_pack 2018-06-13 15:12:42 +01:00
f15f7a674b [dailymotion] add support for password protected videos(closes #9789) 2018-06-13 14:51:19 +01:00
9aca7fe6a3 [abc:iview] Extract more series metadata 2018-06-12 20:25:50 +07:00
e0671819e7 [abc] fix ABC IView extraction and add support for livestreams(closes #16704)(closes #12354) 2018-06-12 13:07:57 +01:00
5d6c81b63f [downloader/http] Fix resume when writing ot stdout (closes #16699) 2018-06-12 03:12:29 +07:00
dc53c78634 [crackle] Add support for sonycrackle.com (closes #16698) 2018-06-12 02:06:30 +07:00
7dc9c60b4b [tvnet] Fix _VALID_URL 2018-06-12 02:05:58 +07:00
e51752754d [tvnet] Improve video id extraction 2018-06-12 01:50:43 +07:00
0645be49cb [inc] PEP 8 2018-06-12 01:41:23 +07:00
a572ae6114 [tvnet] Improve and fix issues (closes #15462) 2018-06-12 01:37:34 +07:00
b2df66aeca [tvnet] Add extractor 2018-06-12 01:37:29 +07:00
93cffb1444 [nrk] Update API hosts and try all previously known ones (closes #16690) 2018-06-11 03:08:36 +07:00
d253df2f65 [wimp] Fix Youtube embeds extraction 2018-06-11 02:40:17 +07:00
e8c6afc168 release 2018.06.11 2018-06-11 01:57:30 +07:00
cc37cc3f99 [ChangeLog] Actualize
[ci skip]
2018-06-11 01:55:16 +07:00
9d581efe05 [npo] Extend _VALID_URL (closes #16682) 2018-06-10 00:26:16 +07:00
ff2e486221 [inc] Add support for another embed schema (closes #16666) 2018-06-09 02:53:04 +07:00
6ae36035d9 [tv4] fix format extraction(closes #16650) 2018-06-06 00:41:08 +01:00
9afd74d705 [nexx] extract free cdn http formats 2018-06-05 01:02:46 +01:00
2e6975306a [nexx] Update tests 2018-06-05 02:59:25 +07:00
06ea7bdd99 [nexx] Add support for free cdn (closes #16538) 2018-06-05 02:55:54 +07:00
d7be705308 [pbs] Add another cove id pattern (closes #15373) 2018-06-05 00:17:26 +07:00
2e190c2ad9 [rbmaradio] Add support for 192k format (closes #16631) 2018-06-04 23:51:25 +07:00
94418c8eb3 release 2018.06.04 2018-06-04 02:41:53 +07:00
f7560859a3 [devscripts/update-copyright] Update copyright year 2018-06-04 02:33:54 +07:00
c6c478f40d [ChangeLog] Actualize
[ci skip]
2018-06-04 02:16:33 +07:00
c3023e9f2e [camtube] Add extractor 2018-06-03 17:09:20 +07:00
77053237c5 [twitter:card] Generalize base API URL 2018-06-03 15:58:12 +07:00
b6b2ccb72f [twitter:card] Extract guest token (closes #16609) 2018-06-03 15:57:45 +07:00
0a10f50e2f [chaturbate] Use geo verification headers 2018-06-03 04:30:33 +07:00
6d155707e6 [bbc] Add support for bbcthree (closes #16612) 2018-06-03 04:07:59 +07:00
eb6793ba97 [youtube] Update tests 2018-06-03 02:23:45 +07:00
7e72694b5e [youtube] Move metadata extraction after video availability check 2018-06-03 02:08:38 +07:00
936784b272 [youtube] Extract track and artist 2018-06-03 02:05:14 +07:00
003fe73ccf [safari] Add support for new URL schema (closes #16614) 2018-06-03 00:53:11 +07:00
1ea559c445 [adn] fix extraction 2018-06-02 18:14:22 +01:00
19e42ead9b release 2018.06.02 2018-06-02 01:51:31 +07:00
73c938e460 [ChangeLog] Actualize
[ci skip]
2018-06-02 01:49:48 +07:00
9b89daefa6 [facebook] Improve extraction (closes #16554) 2018-06-02 01:42:05 +07:00
9d082e7cb8 [facebook] Add support for tahoe player videos (closes #15441)
Specific videos appear to use a newer/different player, this requires a
second request for the video data as the initial request is missing the
specified data.

Additionally these videos have different page content for the uploader
value, which is stored in the `<meta property="og:title"...>` element of
the initial request.
2018-06-02 01:32:53 +07:00
f20f636596 [cbc] Improve extraction (closes #16583, closes #16593) 2018-06-02 00:35:07 +07:00
b995043ab8 Ignore venv directory 2018-06-02 00:18:57 +07:00
85750f8972 [openload] Improve ext extraction 2018-06-02 00:16:22 +07:00
926d97fc6b [9c9media] PEP 8 2018-06-01 05:17:49 +07:00
2593725a9b [twitter:card] Add support for another endpoint (closes #16586) 2018-06-01 05:16:00 +07:00
0bfdcc1495 [openload] Add support for oload.win and oload.download 2018-05-31 22:01:44 +07:00
c3f75e2454 [audimedia] fix extraction(closes #15309) 2018-05-31 12:39:45 +01:00
3a8e3730c1 [francetv] add support for sport.francetvinfo.fr(closes #15645) 2018-05-31 11:40:37 +01:00
acca2ac7f3 [mlb] improve extraction(closes #16587) 2018-05-31 02:50:14 +01:00
128b58ad13 [nhl] remove old extractors 2018-05-31 02:49:35 +01:00
4fd1437d9d [rbmaradio] check formats availability(closes #16585) 2018-05-30 17:08:32 +01:00
e425710554 release 2018.05.30 2018-05-30 21:54:30 +07:00
bc3143ac5e [ChangeLog] Actualize
[ci skip]
2018-05-30 21:52:33 +07:00
e0d42dd4b2 [teamcoco] Fix extraction for full episodes(closes #16573) 2018-05-30 13:21:07 +01:00
a07879d6b2 [spiegel] fix info extraction(#16538) 2018-05-28 00:10:46 +01:00
cfd7f2a636 [apa] Add extractor (closes #15041, closes #15672) 2018-05-27 18:24:54 +07:00
9c65c4a6cd [bellmedia] add support for bnnbloomberg.ca(#16560) 2018-05-27 12:11:53 +01:00
c9e12a618c [9c9media] extract mpd formats and subtitles 2018-05-27 12:10:12 +01:00
8882840ec5 [cammodels] Use geo verification headers 2018-05-26 22:22:58 +07:00
2ce35d9f43 [cammodels] Add another error pattern 2018-05-26 22:22:58 +07:00
f16f48779c [downloader/rtmp] Generalize download messages and report time elapsed on finish 2018-05-26 22:22:58 +07:00
ddd8486a44 [downloader/rtmp] Gracefully handle live streams interrupted by user 2018-05-26 22:22:58 +07:00
68217024e8 remove unnecessary assignment parenthesis 2018-05-26 16:13:54 +01:00
ec2f3d2800 [ufctv] add support for authentication(closes #16542) 2018-05-26 16:13:54 +01:00
8b1da46e8f [cammodels] Improve and simplify (closes #14499) 2018-05-26 21:25:30 +07:00
2a49d01992 [cammodels] Add extractor 2018-05-26 21:25:21 +07:00
261f47306c [utils] fix style id extraction for namespaced id attribute(closes #16551) 2018-05-26 14:38:24 +01:00
c0fd20abca [soundcloud] detect format extension(closes #16549) 2018-05-26 14:38:24 +01:00
986c0b0215 [cbc] Fix playlist title extraction (closes #16502) 2018-05-26 20:05:54 +07:00
97b01144bd [tumblr] Detect and report sensitive media (closes #13829) 2018-05-26 20:00:00 +07:00
56cd31f320 [tumblr] Improve authentication (closes #15133) 2018-05-26 19:59:35 +07:00
c678192af3 [tumblr] Add support for authentication 2018-05-26 19:56:01 +07:00
0934c9d4fa release 2018.05.26 2018-05-26 13:02:21 +07:00
38e4e8ab80 [ChangeLog] Actualize
[ci skip]
2018-05-26 12:58:34 +07:00
5a16c9d9d3 [utils] keep the original TV_PARENTAL_GUIDELINES dict 2018-05-25 23:12:50 +01:00
bdbcc8eecb [dvtv] Remove dead test 2018-05-26 02:15:50 +07:00
9ef5cdb5cb [audiomack] Stringify video id (closes #15310) 2018-05-26 02:13:29 +07:00
03fad17cb6 [izlesene] Improve extraction and fix issues (closes #16407, closes #16271) 2018-05-26 01:53:17 +07:00
f4d261b765 [izlesene] Fix extraction (closes #16233) 2018-05-26 01:53:11 +07:00
aee36ca832 [indavideo] Add support for generic embeds (closes #11989) 2018-05-26 01:25:40 +07:00
2a7c6befc1 [indavideo] Fix extraction (closes #11221) 2018-05-26 01:09:44 +07:00
b39f42ee92 [indavideo] Sign download URLs 2018-05-26 00:46:05 +07:00
6bd499e8ca [peertube] Add support for generic embeds 2018-05-26 00:28:30 +07:00
f2fc63a5a8 [peertube] Add support for embed and API URLs 2018-05-26 00:15:38 +07:00
c561b75c82 [peertube] Add extractor (closes #16301, closes #16329) 2018-05-26 00:09:15 +07:00
3d2a643fdc [imgur] Fix extraction 2018-05-25 21:54:21 +08:00
e8e58c2278 [hidive] add support for authentication(closes #16534) 2018-05-24 11:53:42 +01:00
1139935db7 [nbc] add support for stream.nbcsports.com(closes #13911) 2018-05-24 02:51:47 +01:00
ca0aef42d4 [viewlift] add support for hoichoi.tv(closes #16536) 2018-05-23 23:04:12 +01:00
3bb3ff38a1 [test_utils] add tests for b836118724 2018-05-23 12:20:05 +01:00
268e132dec [go90] extract age limit and detect drm protection(#10127) 2018-05-23 12:15:21 +01:00
670dcba8c7 [viewlift] Remove rating format transformation 2018-05-23 12:13:44 +01:00
b836118724 [utils] Relax TV Parental Guidelines matching 2018-05-23 12:12:20 +01:00
57d6792024 [viewlift] fix extraction for snagfils.com(closes #15766) 2018-05-23 11:27:36 +01:00
b89ac53455 [globo] use compat_str 2018-05-21 17:46:52 +01:00
d81ffc3aa0 [globo] Add entry for netrc authentication 2018-05-21 15:39:02 +01:00
e518749300 [globo] handle login errors 2018-05-21 15:07:24 +01:00
db2058f63e [globo] improve extraction(closes #4189)
- add support for authentication
- simplify url signing
- extract DASH and MSS formats
2018-05-21 14:55:50 +01:00
5c766952dc Update leeco.py
Fixed this bug :youtube_dl.utils.ExtractorError: An extractor error has occurred. (caused by KeyError('location',));
2018-05-21 21:26:53 +08:00
504f20dd30 Remove experimental mark for some options 2018-05-19 23:53:24 +07:00
f2b1fa07ec [teamcoco] relax _VALID_URL regex and add a fallback for format extraction(fixes #16484) 2018-05-19 13:05:51 +01:00
acd620c930 [teamcoco] improve _VALID_URL regex(#16484) 2018-05-19 12:19:05 +01:00
27694fe7ad [imdb:list] fix _VALID_URL regex 2018-05-19 11:04:08 +01:00
0167f0dbfe [imdb] improve extraction(fixes #4085)(fixes #14557) 2018-05-19 10:15:11 +01:00
7550ea501a release 2018.05.18 2018-05-18 00:32:51 +07:00
58197205d3 [ChangeLog] Actualize
[ci skip]
2018-05-18 00:30:41 +07:00
361a965b5c [vimeo:likes] Relax _VALID_URL and fix single page likes extraction (closes #16475) 2018-05-17 23:21:40 +07:00
a3f86160fa [pluralsight] fix clip id extraction(fixes #16460) 2018-05-17 13:46:05 +01:00
1306f5ed72 [mychannels] add support for mychannels.com(closes #15334) 2018-05-16 19:11:48 +01:00
58a68d8fda [moniker] Remove extractor(closes #15336) 2018-05-16 18:44:33 +01:00
eea2fafcf5 [pbs] fix embed data extraction(fixes #16474) 2018-05-16 18:34:25 +01:00
6843ac5b13 add support for paramountnetwork.com and bellator.com(fixes #15418) 2018-05-16 17:49:35 +01:00
54fc90aabf [youtube] fix hd720 format position 2018-05-16 16:24:44 +01:00
997530d9d4 [dailymotion] remove fragment part from m3u8 urls(closes #8915) 2018-05-16 12:04:24 +01:00
fe3a60f040 [dreisat] improve extraction(closes #15350)
- extract all formats
- extract more format metadata extraction
- improve format sorting
- use hls native downloader
- detect geo-restriction
- bypass geo-restriction
2018-05-16 11:30:29 +01:00
7f34984e81 [dtube] Add new extractor(closes #15201) 2018-05-16 09:35:47 +01:00
1e4fe5a7cc [options] Fix typo (closes #16450) 2018-05-14 23:42:33 +07:00
c63ca0eef8 [youtube] Improve format filesize extraction (#16453) 2018-05-14 23:27:56 +07:00
84a9fef899 [youtube] Make uploader extraction non fatal (#16444) 2018-05-13 22:49:01 +07:00
4c76aa0666 [youtube] fix extraction for embed restricted live streams(fixes #16433) 2018-05-13 13:20:16 +01:00
90b633f86b [nbc] improve info extraction(fixes #16440) 2018-05-13 11:31:41 +01:00
07acdc5afc [twitch:clips] Sort formats 2018-05-12 12:08:54 +07:00
49fa7de301 [twitch:clips] Fix extraction (closes #16429) 2018-05-11 23:21:02 +07:00
dbd5c502ea [redditr] Relax _VALID_URL (closes #16426) 2018-05-10 23:17:23 +07:00
bc5e4aa57e [mixcloud] Bypass throttling for HTTP formats (#12579, #16424) 2018-05-10 22:22:26 +07:00
1344d3e169 [nickbr] Relax _VALID_URL (#13230) 2018-05-10 22:01:13 +07:00
ff8889cd4d [teamcoco] fix extraction(closes #16374) 2018-05-10 08:19:56 +01:00
9e18bb4c67 release 2018.05.09 2018-05-09 00:36:47 +07:00
44277998ad [ChangeLog] Actualize
[ci skip]
2018-05-09 00:34:39 +07:00
05108a496a [YoutubeDL] Ensure ext exists for automatic captions 2018-05-08 22:57:52 +07:00
2fbd86352e [udemy] Extract asset captions 2018-05-08 22:57:01 +07:00
0ce76801e8 [udemy] Extract stream URLs (closes #16372) 2018-05-08 22:33:35 +07:00
789b7774a7 [businessinsider] Add extractor (closes #16387, closes #16388, closes #16389) 2018-05-06 21:58:55 +07:00
660a230b2d [cloudflarestream] Add support for cloudflare streams (closes #16375) 2018-05-05 01:21:52 +07:00
a90a6b54ee [watchbox] Fix extraction (closes #16356) 2018-05-02 20:43:34 +07:00
3cc0d0b829 [discovery] extract Affiliate/Anonymous Auth Token from cookies(closes #14954) 2018-05-02 09:32:53 +01:00
ea1f5e5dbd [itv:btcc] Add extractor (closes #16139) 2018-05-02 07:21:24 +07:00
5f95927a62 Improve geo bypass mechanism
* Introduce geo bypass context
* Add ability to bypass based on IP blocks in CIDR notation
* Introduce --geo-bypass-ip-block
2018-05-02 07:20:59 +07:00
a93ce61bd5 [tunein] Use live title for live streams (closes #16347) 2018-05-02 01:29:44 +07:00
c18142da6e [itv] Improve extraction (closes #16253) 2018-05-01 22:48:08 +07:00
cc42941390 release 2018.05.01 2018-05-01 03:38:57 +07:00
cc5772c4f0 [ChangeLog] Actualize
[ci skip]
2018-05-01 03:30:23 +07:00
c21692fa94 [kaltura] Improve iframe embeds detection (closes #16337) 2018-05-01 03:09:04 +07:00
8513963468 [udemy] Extract outputs renditions (closes #16289, closes #16291, closes #16320, closes #16321, closes #16334, closes #16335) 2018-05-01 02:15:43 +07:00
67ca1a8ef7 [zattoo] Improve and simplify (closes #14676) 2018-05-01 01:50:30 +07:00
4a73354586 [zattoo] Add extractor (closes #14668) 2018-05-01 01:50:07 +07:00
796bf9de45 [yandexmusic] Convert release_year to int 2018-04-29 22:56:07 +07:00
e5eadfa82f [udemy,xiami,yandexmusic] Override _download_webpage_handle instead of _download_webpage 2018-04-29 22:54:52 +07:00
30226342ab [youtube] Correctly disable polymer on all requests
Rather than just the one that use the _download_webpage helper. The need
for this was made apparent by 0fe7783e, which refactored
_download_json in a way that completely avoids the use of
_download_webpage, thus breaking youtube.

Fixes #16323
2018-04-29 22:35:16 +07:00
01aec84880 [generic] Prefer enclosures over links in RSS feeds 2018-04-29 22:14:37 +07:00
12b0d4e0e1 [redditr] Add support for old.reddit.com URLs 2018-04-29 21:59:40 +07:00
106c8c3edb [nrktv] Update API host (closes #16324) 2018-04-29 19:04:40 +07:00
500a86a52e [downloader/fragment] Restart download if .ytdl file is corrupt (closes #16312) 2018-04-29 00:33:31 +07:00
7dd6ab4a47 [imdb] Extract all formats (closes #16249) 2018-04-28 04:51:39 +07:00
ae1c585cee [vimeo] Extract JSON LD (closes #16295) 2018-04-28 02:51:18 +07:00
e7e4a6e0f9 [extractor/common] Extract interaction statistic 2018-04-28 02:48:03 +07:00
6cc622327f [utils] Introduce merge_dicts 2018-04-28 02:47:17 +07:00
0fe7783ece [extractor/common] Add _download_json_handle 2018-04-28 01:59:15 +07:00
c84eae4f66 [funk:channel] Improve extraction (closes #16285) 2018-04-27 03:45:52 +07:00
d3711b0050 [devscripts/gh-pages/generate-download.py] Use program checksum from versions.json 2018-04-25 02:14:27 +07:00
b5802d69f5 release 2018.04.25 2018-04-25 01:12:40 +07:00
e028d4f506 [ChangeLog] Actualize
[ci skip]
2018-04-25 01:07:37 +07:00
ecb24f7c08 Credit @f2face for #16115 2018-04-25 01:07:32 +07:00
95284bc281 Credit @TingPing for picarto (#15551) 2018-04-25 01:07:27 +07:00
5d0fe6d23e Credit @Zopieux for #16250 2018-04-25 01:07:23 +07:00
76030543cd [openload] Recognize IPv6 stream URLs (closes #16137) 2018-04-25 00:49:30 +07:00
0ff51adae6 [twitch] Extract is_live according to status (closes #16259) 2018-04-24 23:55:06 +07:00
1cc47c6674 [utils] Fix match_str for boolean meta fields 2018-04-24 23:54:49 +07:00
99036a1298 [pornflip] Relax _VALID_URL (closes #16258) 2018-04-23 04:03:11 +07:00
171625469a [etonline] Remove extractor (closes #16256)
Covered by generic extractor
2018-04-23 03:17:34 +07:00
af751350e8 [Makefile] Add support for pandoc 2 and disable smart extension (closes #16251)
smart extension rewrites straight quotes as curly quotes, -- as en-dashes and so on that is unwanted behavior.
2018-04-23 02:50:11 +07:00
2441c1aab1 [breakcom] Fix extraction (closes #16254) 2018-04-23 00:16:52 +07:00
70d35d166c [youtube] Add ability to authenticate with cookies 2018-04-22 06:08:05 +07:00
3853309fe2 [youtube:feed] Implement lazy playlist extraction (closes #10184) 2018-04-22 06:07:32 +07:00
6cdaaf7031 [svt] Improve (closes #15809) 2018-04-22 05:34:03 +07:00
488ff2dd3a [svt] Add support for TV channel live streams (Closes #15279) 2018-04-22 05:33:40 +07:00
353f0bde78 [cbssports] PEP 8 2018-04-22 04:57:22 +07:00
040c6296bb [ccma] Fix video extraction (closes #15931) 2018-04-22 04:55:35 +07:00
a693386df1 [rentv] Improve extraction (closes #15227) 2018-04-21 23:22:30 +07:00
4b8588fe02 [rentv] Fix extraction 2018-04-21 23:22:25 +07:00
d65a48a0ef [nick] Add support for nickjr.nl (closes #16230) 2018-04-20 23:12:13 +07:00
c194200277 [mofosex] Fix test 2018-04-19 22:38:31 +07:00
d317973284 [extremetube] Fix metadata extraction 2018-04-19 22:36:33 +07:00
1792bc3a06 [keezmovies] Add support for generic embeds (closes #16134) 2018-04-19 22:25:51 +07:00
5a19d231ca [YoutubeDL] Fix typo in media extension compatibility checker 2018-04-19 22:21:50 +07:00
d86c5167ae [nexx] extract new azure urls(closes #16223) 2018-04-19 15:48:03 +01:00
b004d9bbf1 [cbssports] fix extraction(fixes #16217) 2018-04-19 15:08:17 +01:00
9b3036bd2e [instagram:user] Fix extraction (closes #16119) 2018-04-18 10:12:24 +07:00
e30991f920 [kaltura] Improve embeds detection (closes #16201) 2018-04-18 01:26:15 +07:00
518d5ba519 Fix some tests 2018-04-18 00:10:02 +07:00
238d42cf5d [instagram:user] Fix extraction (closes #16119) 2018-04-17 22:37:50 +07:00
522d6b5c96 [cbs] skip DRM asset types(fixes #16104) 2018-04-16 07:48:51 +01:00
3c92fd1cd5 release 2018.04.16 2018-04-16 01:09:18 +07:00
bdf7ba6f3a Set chmod 644 for all extractors 2018-04-16 01:07:21 +07:00
0e6ccb3905 [ChangeLog] Actualize
[ci skip]
2018-04-16 00:56:05 +07:00
c07cb68e79 [smotri:broadcast] Fix extraction (closes #16180) 2018-04-16 00:54:21 +07:00
a42839e548 [picarto] Improve extraction (closes #6205, closes #12514, closes #15276, closes #15551) 2018-04-16 00:34:47 +07:00
d6166a7602 [picarto] Add extractor 2018-04-16 00:32:15 +07:00
8e41c9ad01 [vine:user] Improve extraction (closes #16190) 2018-04-15 22:46:43 +07:00
9b5aead6aa [vine:user] Fix extraction (closes #15514) 2018-04-15 22:46:30 +07:00
68ddba20ae [instagram:user] Remove User-Agent from signature (closes #16119) 2018-04-13 22:28:33 +07:00
92ded33a05 [pornhub] Relax _VALID_URLs (closes #16165) 2018-04-12 04:53:45 +07:00
64f03e5b4c [cbc:watch] Re-acquire device token when expired (closes #16160) 2018-04-11 23:30:19 +07:00
d783aee56a [fxnetworks] Add support for https theplatform URLs (closes #16125) 2018-04-11 20:11:24 +07:00
315ab3d500 [instagram:user] Simplify signing (#16119) 2018-04-11 01:51:57 +07:00
dd9aea8cbd [instagram:user] Add request signing (closes #16119) 2018-04-11 01:25:41 +07:00
fce7962691 [twitch] Add support for mobile URLs (closes #16146) 2018-04-10 23:07:37 +07:00
f7f9757efc release 2018.04.09 2018-04-09 01:19:27 +07:00
880ed89d49 [ChangeLog] Actualize
[ci skip]
2018-04-09 01:14:47 +07:00
94c3442e6a [YoutubeDL] Do not save/restore console title while simulate (closes #16103) 2018-04-09 01:04:22 +07:00
069937151e [generic] Add support for tube8 embeds 2018-04-09 00:37:15 +07:00
d3431dcb90 [generic] Restrict share-videos.se embeds regex to filter bogus URLs (#16115) 2018-04-09 00:25:44 +07:00
1fc37ca3f1 [generic] Add support for share-videos.se embeds (closes #16089) 2018-04-09 00:19:23 +07:00
d04ca97616 [odnoklassniki] Improve _VALID_URL readability 2018-04-08 22:21:21 +07:00
608c738c7d [odnoklassniki] Extend _VALID_URL (closes #16081) 2018-04-08 22:13:00 +07:00
66b686727b [extractor/common] Relax JSON-LD context check (closes #16006) 2018-04-08 03:09:42 +07:00
717ea4e14e [steam] Bypass mature content check (closes #16113) 2018-04-08 00:29:43 +07:00
cae5d9705c [acast] Extract more metadata 2018-04-08 00:21:55 +07:00
1c9b1a4494 [acast] Fix extraction (closes #16118) 2018-04-08 00:08:45 +07:00
ff826177cc [instagram:user] Fix extraction (closes #16119) 2018-04-07 23:58:57 +07:00
9d15be3a5b [drtuber] Fix title extraction (closes #16107) 2018-04-07 21:39:21 +07:00
e2750e1437 [liveleak] Extend _VALID_URL (closes #16117) 2018-04-07 20:55:01 +07:00
e944737c59 [openload] Add support for oload.xyz 2018-04-06 23:40:15 +07:00
fdfb32a0dd [openload] Relax stream URL regex 2018-04-06 00:15:22 +07:00
235d828b7b [openload] Fix extraction (closes #16099) 2018-04-05 23:49:15 +07:00
1236ac6b0b [svtplay] Share svtplay regex 2018-04-05 00:29:13 +07:00
df146eb282 [svtplay:series] Add support for season URLs 2018-04-05 00:29:08 +07:00
b71bb3ba8b [svtplay:series] Improve extraction (closes #16059) 2018-04-05 00:29:02 +07:00
fd97fa7bfc [svtplay:series] Add extractor
Related to #11130
2018-04-05 00:28:58 +07:00
e8dfecb384 release 2018.04.03 2018-04-03 00:26:11 +07:00
10f9caec04 [ChangeLog] Actualize
[ci skip]
2018-04-03 00:23:03 +07:00
ea6679fbeb [tvnow] Fix issues, simplify and improve (closes #15837) 2018-04-03 00:08:22 +07:00
3acae1e031 [tvnow] Add support for shows 2018-04-03 00:06:47 +07:00
8bd1df3c31 [dramafever] Fix authentication (closes #16067) 2018-04-02 22:19:42 +07:00
86693c4930 [afreecatv] Use partial view only when necessary (closes #14450) 2018-04-02 00:00:45 +07:00
d563fb32ba [afreecatv] Remove debug output 2018-04-01 23:07:54 +07:00
e51762be19 [afreecatv] Add support for authentication (#14450) 2018-04-01 22:47:39 +07:00
03fcde10ce [nationalgeographic] Add support for new URL schema (closes #16001) 2018-04-01 21:22:51 +07:00
95a1322bc1 [bilibili] Remove debug from player params regexes 2018-04-01 02:06:58 +07:00
0669f8fd8f [xvideos] Fix thumbnail extraction (closes #15978) 2018-03-31 23:46:08 +07:00
0b4bbcdcb6 [medialaan] Fix vod id 2018-03-31 22:14:49 +07:00
3e78d23b57 [openload] Add support for oload.site 2018-03-30 23:25:43 +07:00
190f6c936b [naver] Fix extraction (closes #16029) 2018-03-29 23:49:09 +07:00
02f6ccbce3 [dramafever] Partially switch to API v5 (closes #16026) 2018-03-29 23:06:13 +07:00
5d60b99717 [options] Mention comments support in --batch-file 2018-03-27 22:25:29 +07:00
9e6a418015 [abc:iview] Unescape title and series meta fields 2018-03-27 22:08:40 +07:00
99c3091850 [videa] Extend _VALID_URL 2018-03-27 22:02:04 +07:00
bbd9d8c170 release 2018.03.26.1 2018-03-26 22:32:03 +07:00
c3cfc71a0c [ChangeLog] Actualize
[ci skip]
2018-03-26 22:30:11 +07:00
671e241bfb release 2018.03.26 2018-03-26 05:03:47 +07:00
29d9594561 [ChangeLog] Actualize
[ci skip]
2018-03-26 22:11:01 +07:00
f0298f653e [downloader/external] Simplify finished progress hook reporting and add elapsed time (closes #10876) 2018-03-24 16:35:21 +07:00
2ea212628e [downloader/common] Improve progress reporting when no total bytes available 2018-03-24 16:35:15 +07:00
80aa246094 [downloader/external] Fix download finalization when writing file to stdout (closes #10809)
An OSError or IOError generally indicates something a little more
wrong than a "simple" UnavailableVideoError, so print the actual
traceback that leads to the exception. Otherwise meaningful postmortem
debugging a bug report is essentially infeasible.
2018-03-24 16:34:55 +07:00
0ff2c1ecb6 [downloader/fragment] Fix download finalization when writing file to stdout (closes #15799) 2018-03-24 16:04:23 +07:00
16132cff72 [vrv] Fix extraction on python2 (closes #15928) 2018-03-24 14:57:34 +07:00
86e1958944 [afreecatv] Update referrer (closes #15947) 2018-03-24 14:21:08 +07:00
b015cb1af3 [24video] Add support for 24video.sexy (closes #15973) 2018-03-24 14:11:27 +07:00
7d34016fb0 [crackle] Bypass geo restriction 2018-03-24 01:49:50 +07:00
b9f5a41207 [crackle] Fix extraction (closes #15969) 2018-03-23 23:53:18 +07:00
8b7340a45e [lenta] Add extractor (closes #15953) 2018-03-22 23:07:31 +07:00
1d4a0520ba Merge pull request #15939 from sudovijay/patch-11
[Youku] Update ccode
2018-03-22 14:42:35 +08:00
cba5d1b6b3 [instagram:user] Add pagination (closes #15934) 2018-03-21 23:43:03 +07:00
328ddf56a1 [Youku] Update ccode 2018-03-21 12:13:31 +05:30
3395958d2b libsyn: adapt to new page structure and replace testcase 2018-03-20 23:07:11 +01:00
a66d1d079a release 2018.03.20 2018-03-20 01:55:48 +07:00
c651de39d5 [ChangeLog] Actualize
[ci skip]
2018-03-20 01:54:35 +07:00
d9e2240f7c [7plus] Extract series metadata (closes #15862, closes #15906) 2018-03-20 01:40:53 +07:00
832f9d5258 [9now] Bypass geo restriction (closes #15920) 2018-03-20 01:08:03 +07:00
21dedcb580 [cbs] Skip unavailable assets (closes #13490, closes #13506, closes #15776) 2018-03-20 00:27:39 +07:00
6780154e6b [extractor/common] Improve thumbnail extraction for HTML5 entries 2018-03-19 23:43:53 +07:00
38f59e2793 [canalc2] Add support for HTML5 videos (closes #15916, closes #15919) 2018-03-19 23:40:19 +07:00
9a054fcbba [ceskatelevize] Add support for iframe embeds (closes #15918) 2018-03-19 23:29:53 +07:00
6e3f23d912 [prosiebensat1] Add support for galileo.tv (closes #15894) 2018-03-19 04:14:33 +07:00
47a5cb7734 Generalize XML manifest processing code and improve XSPF parsing (closes #15794) 2018-03-18 02:52:17 +07:00
e0d198c18d [extractor/common] Add _download_xml_handle 2018-03-18 02:52:01 +07:00
96b8b9abae [extractor/generic] Support relative URIs in _parse_xspf
<location> can have relative URIs, not just absolute.
2018-03-18 02:48:44 +07:00
178ee88319 [generic] Add support for xfileshare embeds (closes #15879) 2018-03-17 23:57:07 +07:00
d123960857 [bilibili] Switch to v2 playurl API 2018-03-16 03:18:53 +07:00
3526c3043b [bilibili] Fix and improve extraction (closes #15048, closes #15430, closes #15622, closes #15863) 2018-03-16 00:19:17 +07:00
8e70c1bfac [heise] Improve extraction (closes #15496, closes #15784, closes #15026) 2018-03-15 23:09:24 +07:00
27b1c73f14 [instagram] fix user videos extraction(fixes #15858) 2018-03-15 14:33:36 +01:00
46c6742d4f release 2018.03.14 2018-03-14 22:49:22 +07:00
c95dfb0509 [ChangeLog] Actualize
[ci skip]
2018-03-14 22:45:05 +07:00
b8c6badc96 [soundcloud] Update client id (closes #15866) 2018-03-14 22:38:55 +07:00
b848a4ca1a [tennistv] Remove duplicate key in dictionary 2018-03-14 11:48:26 +01:00
e6e68069f6 [tennistv] Correctly encode POST parameters
In python 3.x, the POST parameters must be bytes, not str.
2018-03-14 11:23:10 +01:00
f3672ac522 [line] lint (remove space on empty line) 2018-03-14 09:55:46 +01:00
f226880c6d [tennistv] Add support for tennistv.com 2018-03-14 09:55:21 +01:00
08250b69c2 [line] Add support for tv.line.me (closes #9427) 2018-03-11 22:25:30 +08:00
d116918993 [raywenderlich] PEP 8
[ci skip]
2018-03-10 23:15:11 +07:00
7399ca1f80 [pornhub] PEP 8
[ci skip]
2018-03-10 23:14:37 +07:00
b4a190fe2a [xnxx] Fix extraction (closes #15817) 2018-03-10 23:13:14 +07:00
cc7f6c720e [njpwworld] Fix authentication (closes #15815) 2018-03-10 22:35:47 +07:00
3a0ceb32e2 release 2018.03.10 2018-03-10 04:45:57 +07:00
7dee417127 [ChangeLog] Actualize
[ci skip]
2018-03-10 04:44:46 +07:00
5b1d158834 [raywenderlich] Extract videos in order 2018-03-10 04:31:51 +07:00
a7298f3e99 [pornhub] Don't override session cookies (closes #15697) 2018-03-09 23:57:32 +07:00
5d49d879cc [raywenderlich] Add extractor (#15251) 2018-03-09 23:27:44 +07:00
b5434b5c31 [nexx] Fix typo 2018-03-08 03:25:04 +07:00
690404a6f8 [funk] Fix extraction and rework extractors (closes #15792) 2018-03-08 03:17:46 +07:00
d91dd0ce19 [nexx] Restore reverse engineered approach 2018-03-08 03:16:21 +07:00
6202f08e1b [heise] Add support for kaltura embeds (closes #14961) 2018-03-06 23:10:01 +07:00
574e9db2b0 [tvnow] Extract series metadata (closes #15774) 2018-03-06 23:06:00 +07:00
2e25f80d5d [ruutu] Continue formats extraction on NOT-USED URLs 2018-03-06 02:01:04 +07:00
64f34528df [vrtnu] Use redirect URL for building video JSON URL (closes #15767, closes #15769) 2018-03-05 22:57:19 +07:00
26ad6bcdfc [vimeo] Modernize login code and improve error messaging 2018-03-05 22:45:47 +07:00
81dc74966a [archiveorg] Fix extraction (closes #15770, closes #15772) 2018-03-05 22:30:32 +07:00
d53b6764d0 [hidive] Remove proxy from params 2018-03-04 23:23:30 +07:00
62f49dd3b9 [hidive] Add extractor (closes #15494) 2018-03-04 17:46:36 +07:00
f9f10268c1 [afreecatv] Detect deleted videos 2018-03-04 03:13:45 +07:00
f241a97312 [afreecatv] Fix extraction (closes #15755) 2018-03-04 03:01:58 +07:00
86c8cfc555 [vice] Fix extraction and rework extractors (closes #11101, closes #13019, closes #13622, closes #13778) 2018-03-03 23:08:43 +07:00
c01db237b5 [vidzi] Add support for vidzi.si (closes #15751) 2018-03-03 20:16:55 +07:00
0093c77032 [downloader/hls] Skip uplynk ad fragments (closes #15748) 2018-03-03 20:00:25 +07:00
5616caf852 [npo] Fix typo 2018-03-03 01:47:09 +07:00
05a7ffb126 release 2018.03.03 2018-03-03 01:37:01 +07:00
28f21c9501 [ChangeLog] Actualize
[ci skip]
2018-03-03 01:32:21 +07:00
4c780fbd0a [yapfiles] Add extractor (closes #15726, refs #11085) 2018-03-03 01:24:36 +07:00
7773a92800 [spankbang] Fix formats extraction (closes #15727) 2018-03-02 23:39:20 +07:00
b871d7e954 [utils] Add parse_resolution 2018-03-02 23:39:04 +07:00
44dc11db61 [adn] fix format extraction(#15716) 2018-02-28 19:41:30 +01:00
949faa15e8 [toggle] Extract DASH and ISM formats (closes #15721) 2018-02-28 22:55:09 +07:00
0c3e5f4921 Revert "Respect --prefer-insecure while updating (closes #15497)"
This reverts commit 7d2b4aa047.
2018-02-27 22:30:08 +07:00
266fbd6b73 [nickelodeon] Add support for nickelodeon.com.tr (closes #15706) 2018-02-26 22:10:44 +07:00
d1b6187012 [npo] Validate and filter format URLs (closes #15709) 2018-02-26 21:50:51 +07:00
6ab35f5e16 release 2018.02.26 2018-02-26 04:23:38 +07:00
32ae31847f [ChangeLog] Actualize 2018-02-26 04:19:04 +07:00
abe8766c35 [udemy] Use custom User-Agent (closes #15571) 2018-02-26 04:12:53 +07:00
eaa3172672 release 2018.02.25 2018-02-25 20:38:10 +07:00
797c9284d6 [ChangeLog] Actualize 2018-02-25 20:35:52 +07:00
8c73ef37b6 [vidlii] Add extractor (closes #14472, closes #14512, closes #14779) 2018-02-25 20:28:40 +07:00
b5cbe3d652 [postprocessor/embedthumbnail] Skip embedding when there aren't any thumbnails 2018-02-25 19:33:13 +07:00
ece12e6348 [streamango] Skip dead test 2018-02-25 18:36:25 +07:00
ff274e3c16 [streamango] Capture and output error messages 2018-02-25 18:34:52 +07:00
c106237d56 [streamango] Fix formats extraction, improve and simplify (closes #14256) 2018-02-25 18:27:23 +07:00
6e72ea4775 [streamango] Fix extraction (closes #14160) 2018-02-25 18:26:48 +07:00
d6a0350253 [ard] Remove dead tests 2018-02-25 17:41:12 +07:00
ad29ef043e [ard] Add alive tests 2018-02-25 17:38:07 +07:00
f01df14c4f [telequebec:emission] Extend _VALID_URL 2018-02-25 17:05:39 +07:00
9306b0c8d9 [telequebec] Add support for emissions and refactor (closes #14649, closes #14655) 2018-02-25 16:54:12 +07:00
f4b7427279 [extractor/common] Improve jwplayer subtitles extraction (closes #15695) 2018-02-25 00:59:29 +07:00
300148b48a [telequebec:live] Add extractor (closes #15688) 2018-02-24 06:17:29 +07:00
2d17c63140 [abcnews] Update tests 2018-02-24 05:17:21 +07:00
f2908d072e [mailru:music] Add extractor (closes #15618) 2018-02-24 04:52:55 +07:00
5e7841932c [aenetworks] switch to akamai hls formats(closes #15612) 2018-02-23 08:23:55 +01:00
870f3bfc63 [ytsearch] Fix flat title extraction (closes #11260, closes #15681) 2018-02-23 03:43:42 +07:00
3d977fe4d2 release 2018.02.22 2018-02-22 23:50:35 +07:00
f075838728 [ChangeLog] Actualize 2018-02-22 23:48:58 +07:00
2acc11d771 [vidio] Fix HLS URL extraction (closes #15675) 2018-02-22 22:50:39 +07:00
0704306e1d [nexx] Add support for arc.nexx.cloud URLs 2018-02-22 22:31:28 +07:00
9dc7ea320d [nexx] Don't capture domain id and add support for domainless shortcuts 2018-02-22 22:27:19 +07:00
e231afb14f [nexx] switch to ark api(closes #15652) 2018-02-22 10:41:47 +01:00
12acb9a6fb [zdf] Update tests 2018-02-21 21:57:34 +07:00
18ebd1a843 [redtube] Fix duration extraction and update test 2018-02-21 21:55:28 +07:00
8315ee6c4c [reddit] Update test 2018-02-21 04:12:56 +07:00
b9d1a79426 [9gag] Update test 2018-02-20 22:28:54 +07:00
09f934b009 [vk] Update test 2018-02-20 22:21:10 +07:00
73af6e22fd [vimeo] Update test 2018-02-20 22:20:15 +07:00
77e499f95e [xhamster] Update test 2018-02-20 22:18:50 +07:00
befa4708fd [utils] Fixup some common URL's typos in sanitize_url (closes #15649) 2018-02-19 22:50:23 +07:00
90830004c8 [sonyliv] Respect referrer (closes #15648) 2018-02-19 22:29:08 +07:00
18d7aa6efa [brightcove:new] Use referrer for formats' HTTP headers 2018-02-19 22:28:27 +07:00
b12cf31bb1 [cbc] add new extractor for olympics.cbc.ca(closes #15535) 2018-02-19 09:02:23 +01:00
7d2b4aa047 Respect --prefer-insecure while updating (closes #15497) 2018-02-18 16:43:54 +07:00
38662dfec7 [fusion] Add support for fusion.tv 2018-02-17 20:54:52 +07:00
ee706f1009 [npo] Improve quality metadata extraction 2018-02-17 20:32:34 +07:00
c4e7496421 [npo] Relax _VALID_URL (closes #14987, closes #14994) 2018-02-17 20:32:26 +07:00
b8adcec4ea [npo] Capture and output error message 2018-02-17 20:32:20 +07:00
073cca3df8 [downloader/common] Add whitespace 2018-02-17 19:11:46 +07:00
f66df20ccd [pornhub] Add support for channels (closes #15613) 2018-02-17 01:17:06 +07:00
ea69624992 [youtube] Handle shared URLs with generic extractor (closes #14303) 2018-02-15 22:33:11 +07:00
49702e3669 [francetv] Fix typo 2018-02-12 00:25:42 +07:00
59b5e7b280 release 2018.02.11 2018-02-11 22:31:52 +07:00
b9683400cf [ChangeLog] Actualize 2018-02-11 22:16:42 +07:00
760f81212f [francetv] Add support for live streams (closes #13689) 2018-02-11 21:58:04 +07:00
79080573b5 [francetv] Add support for zouzous.fr and ludo.fr (closes #10454, closes #13087, closes #13103, closes #15012) 2018-02-11 21:40:38 +07:00
99892e9908 [francetv] Separate main extractor and rework others to delegate to it 2018-02-11 21:14:05 +07:00
8faa338ff3 [francetv] Improve manifest URL signing (closes #15536) 2018-02-11 20:06:37 +07:00
818df33fda [francetv] Sign m3u8 manifest URLs (closes #15565)
they seem to be required now, else we get 403 errors.
2018-02-11 20:06:37 +07:00
a072a12e24 [veoh] Add support for embed URLs 2018-02-11 06:51:10 +07:00
e67734dda9 [dvtv] Skip download on failing test 2018-02-11 04:56:03 +07:00
9e36fedd00 [afreecatv] Fix extraction (closes #15556) 2018-02-10 22:36:26 +07:00
d2b200eef9 [periscope] Use accessVideoPublic endpoint (closes #15554) 2018-02-10 17:23:37 +07:00
a03a3c80fe [YoutubeDL] Add support for filesize_approx in format selector (closes #15550) 2018-02-10 16:42:45 +07:00
9d5871fdff [discovery] Fix auth request (closes #15542) 2018-02-10 00:55:11 +07:00
ff873b5777 [6play] Extract subtitles (closes #15541) 2018-02-10 00:04:08 +07:00
042968ff31 Credit @mweinelt for #15124 2018-02-09 23:10:36 +07:00
430f2ca544 Credit @iamleot for internazionale (#14973) 2018-02-09 23:10:30 +07:00
cbfbf07cdc Credit @che0 for seznamzpravy (#14616) and dvtv (#15442) 2018-02-09 23:10:24 +07:00
9e167e1ee3 [newgrounds] Fix metadata extraction (closes #15531) 2018-02-09 21:17:02 +07:00
5828489072 [nbc] add support for NBC Olympics Streams(closes #10295) 2018-02-09 02:05:28 +01:00
8c5fafe29f [dvtv] Simplify (closes #15442) 2018-02-09 02:19:29 +07:00
6f4ad0db34 [dvtv] Fix live streams extraction
(also fixed test broken by changed og tags on the site)
2018-02-09 02:19:05 +07:00
c2b3bd0451 release 2018.02.08 2018-02-08 23:39:40 +07:00
728cee5385 [ChangeLog] Actualize 2018-02-08 23:37:49 +07:00
246a75b4ff [extractors] Import for myvi:embed 2018-02-08 23:05:13 +07:00
4fac463d70 [pokemon] PEP 8 2018-02-08 23:04:20 +07:00
382b8182ce [gameinformer] PEP 8 2018-02-08 23:03:57 +07:00
ce53320b11 [myvi] Extend _VALID_URL 2018-02-08 23:02:05 +07:00
51b0557d1e [myvi:embed] Add extractor (closes #15521) 2018-02-08 22:57:33 +07:00
5a5860825d [prosiebensat1] Extend _VALID_URL (closes #15520) 2018-02-08 22:28:56 +07:00
237d07f114 [pokemon] Relax _VALID_URL and extend title extraction (closes #15518) 2018-02-08 03:58:35 +07:00
9f4ec3de25 [gameinformer] Use geo verification headers 2018-02-07 21:52:15 +07:00
96a0bbdd0d [la7] Fix extraction (closes #15501) 2018-02-07 21:48:16 +07:00
c8064d4fab [gameinformer] Fix brightcove id extraction 2018-02-07 21:40:48 +07:00
fde677fed4 [afreecatv] Pass referrer to video info request (closes #15507) 2018-02-06 22:21:10 +07:00
0e0508c8a2 [telebruxelles] Relax _VALID_URL and add support for live streams 2018-02-05 23:56:00 +07:00
bcf150e435 [telebruxelles] Fix extraction (closes #15504) 2018-02-05 23:56:00 +07:00
240f26229d [extractor/common] Respect secure schemes in _extract_wowza_formats 2018-02-05 23:56:00 +07:00
b9b150def7 release 2018.02.04 2018-02-04 08:03:00 +07:00
d20225f33b [ChangeLog] Actualize 2018-02-04 08:01:09 +07:00
5399ab3f0c [brightcove] Pass embed page URL as referrer (closes #15486) 2018-02-04 07:55:04 +07:00
b91a7a4e5e [downloader/http] Randomize HTTP chunk size 2018-02-04 07:33:18 +07:00
e4a60912b8 [youtube] Enforce using chunked HTTP downloading for DASH formats 2018-02-04 07:17:26 +07:00
00c97e3e7a [downloader/http] Add ability to pass downloader options via info dict 2018-02-04 07:16:22 +07:00
cf7259bc93 [downloader/http] Fix 302 infinite loops by not reusing requests 2018-02-04 06:58:34 +07:00
b54d4a5ce8 Document http_chunk_size 2018-02-04 02:53:50 +07:00
db157d2a2a release 2018.02.03 2018-02-03 23:36:03 +07:00
6fcc053947 [test_downloader_http] Use try_rm 2018-02-03 23:24:27 +07:00
a3e8146ea8 [ChangeLog] Actualize 2018-02-03 23:21:18 +07:00
f19eae429a [test_http] Use 127.0.0.1 instead of localhost 2018-02-03 23:09:25 +07:00
ba515388b8 Introduce --http-chunk-size 2018-02-03 23:08:58 +07:00
e2e18694db [redbulltv] fix extraction(closes #15481) 2018-02-03 15:43:33 +01:00
4989d351b4 flake8: Ignore E741 'ambiguous variable name'
Most of violating codes are reverse-engineered JavaScripts. IMO it's
better to keep original (obfuscated) names.

[skip ci]
2018-02-03 21:02:30 +08:00
1367c798e3 [redtube] Fix metadata extraction (closes #15472) 2018-02-02 22:32:53 +07:00
9a340af37e [compat] Mute some F821 under python 3 2018-02-02 03:18:22 +07:00
3c3bceb41d [pladform] Respect platform id and extract HLS formats (closes #15468) 2018-02-02 03:07:30 +07:00
64a12edb48 [rtlnl] Remove progressive formats (closes #15459) 2018-02-01 21:30:17 +07:00
4bf18702e6 [6play] do no modify asset urls with a token(#15248) 2018-02-01 11:22:05 +01:00
ecc218ab14 [nationalgeographic] Relax _VALID_URL 2018-01-31 09:19:11 +01:00
d6b152915c [dplay] Relax _VALID_URL (closes #15458) 2018-01-31 01:29:00 +07:00
69a934e9ad [ISSUE_TEMPLATE_tmpl.md] Add entry on checking URLs availability 2018-01-30 23:31:41 +07:00
5fa2a6a561 [soundcloud] Remove unused _IPHONE_CLIENT_ID 2018-01-30 22:51:43 +07:00
d2a422f548 [cbsinteractive] fix data extraction(closes #15451) 2018-01-29 15:33:24 +01:00
b9d52fb2ca [seznamzpravy] Remove debug output 2018-01-29 04:40:07 +07:00
466000fc6b [amcnetworks] add support for SundanceTV(closes #9260) 2018-01-28 11:30:20 +01:00
65220c3bd6 Add support for IronPython 2018-01-28 05:48:18 +07:00
c989bdbef8 [downloader/ism] Fix Python 3.2 support 2018-01-28 05:20:21 +07:00
eee1692ff3 release 2018.01.27 2018-01-27 23:44:28 +07:00
07e56e6df7 [ChangeLog] Actualize 2018-01-27 23:41:25 +07:00
3c3a07ee0b [seznamzpravy] Improve and simplify (closes #14616) 2018-01-27 23:36:44 +07:00
27940ca09c [seznamzpravy] Add extractor (closes #14102) 2018-01-27 23:34:31 +07:00
3931b84597 [extractor/common] Improve _json_ld for articles 2018-01-27 23:24:38 +07:00
a0ee342b50 [dplay] Bypass geo restriction 2018-01-26 23:56:31 +07:00
864a4576b7 [dplay] Add support for disco-api videos (closes #15396) 2018-01-26 23:49:47 +07:00
bbb7c3f7e9 [youtube] Extract precise error messages (closes #15284) 2018-01-25 22:30:33 +08:00
9d6458a206 [teachertube] Capture and output error message 2018-01-24 22:46:04 +07:00
837b061710 [teachertube] Fix and relax thumbnail extraction (closes #15403) 2018-01-24 22:41:25 +07:00
967ebbdb6c [prosiebensat1] add another clip ID regexp(fixes #15378) 2018-01-23 19:22:44 +01:00
dc400ed6a2 [tbs] update tokenizer url(fixes #15395) 2018-01-23 19:06:46 +01:00
cf2820710d Switch codebase to use compat_b64decode 2018-01-23 22:23:12 +07:00
5d7d805ca9 [mixcloud] Use compat_b64decode (closes #15394) 2018-01-23 21:53:45 +07:00
f206126df0 [compat] Add compat_b64decode 2018-01-23 21:53:01 +07:00
021bd012bb [thesixtyone] Remove extractor (closes #15341) 2018-01-22 22:30:28 +07:00
6e5eacb770 release 2018.01.21 2018-01-21 21:26:05 +07:00
d7da6db4e1 [ChangeLog] Actualize 2018-01-21 21:23:24 +07:00
721a0c3c7b [prosiebensat1] Relax clip id 2018-01-21 21:22:38 +07:00
e0ab56571e [southparkdk] Add support for southparkstudios.nu 2018-01-21 18:42:34 +07:00
99d6e696fc [southpark] Add tests for collections (closes #14803) 2018-01-21 18:41:46 +07:00
6289e07883 [southpark] Add support for collections 2018-01-21 18:41:10 +07:00
655c410063 [test_download] Fix download tests for lazy extractors (closes #13554, closes #13757) 2018-01-21 18:15:11 +07:00
b2a027fc6f [franceinter] Fix upload date extraction (closes #14996) 2018-01-21 17:50:53 +07:00
0d9c48de4f [extractor/common] Improve DASH formats extraction for jwplayer (#9242, #15187) 2018-01-21 17:42:48 +07:00
df58ecbeba [rtvs] Add extractor (closes #9242, closes #15187) 2018-01-21 17:40:23 +07:00
ac458e90a3 [restudy] Extend _VALID_URL (#15347) 2018-01-21 01:22:16 +07:00
7df18fcc65 [restudy] Fix extraction (closes #15347) 2018-01-20 23:19:02 +07:00
c707b1d828 [test_utils] Add tests for malformed JSON handling in js_to_json 2018-01-20 23:00:09 +07:00
c384d537f8 [util] Improve scientific notation handling in js_to_json (closes #14789) 2018-01-20 22:54:21 +07:00
e7f3529f68 [youtube:live] Improve live detection (closes #15365) 2018-01-20 17:57:20 +07:00
7d5406216a [springboardplatform] Add extractor 2018-01-20 00:34:10 +07:00
2a3683c378 prosiebensat1: add another clip ID regexp 2018-01-19 18:26:47 +01:00
154e4fdace [ringtv] Remove extractor (closes #15345) 2018-01-19 22:49:58 +07:00
e2fc6df169 release 2018.01.18 2018-01-18 23:41:44 +07:00
68da3d033c [ChangeLog] Actualize 2018-01-18 23:39:15 +07:00
67408fe0e9 [soundcloud] Update client id (closes #15306) 2018-01-18 22:30:43 +07:00
cad9caf76b [kamcord] Remove extractor (closes #15322) 2018-01-18 22:26:43 +07:00
4471affc34 [spiegel] Add support for nexx videos (closes #15285) 2018-01-17 22:03:56 +07:00
1370dba59f [twitch] Fix authentication and error capture (closes #14090, closes #15264) 2018-01-16 22:34:16 +07:00
1d1d60f6dd [vk] Detect more errors due to copyright complaints (#15259) 2018-01-16 00:51:50 +07:00
a86922c470 [README.md] Clarify macOS name 2018-01-14 00:58:38 +07:00
e11ccd76c6 release 2018.01.14 2018-01-14 00:13:56 +07:00
dd896a6a07 [ChangeLog] Actualize 2018-01-14 00:10:04 +07:00
391dd6f094 [youtube] Fix live streams extraction (closes #15202) 2018-01-14 00:03:22 +07:00
0ce39bc542 [wdr] Fix test 2018-01-13 23:33:52 +07:00
1915662d4f [wdr] Bypass geo restriction 2018-01-13 23:30:56 +07:00
54e8f62e01 [wdr] Rework extractors (closes #14598) 2018-01-13 23:30:25 +07:00
2d8bb80c60 [wdr:elefant] Add extractor 2018-01-13 23:29:36 +07:00
df16e645f6 [gamestar] Fix issues (closes #15179) 2018-01-13 19:38:58 +07:00
d4aedca3bd [gamestar] Add support for gamepro.de (closes #3384) 2018-01-13 19:36:59 +07:00
47e2a9bc53 [viafree] Skip rtmp formats (closes #15232) 2018-01-13 18:47:47 +07:00
e565a6386e Credit @scil for ximalaya extractor (#14687)
[ci skip]
2018-01-12 15:36:01 +08:00
609850acfb [pandoratv] Add support for mobile URLs (closes #12441) 2018-01-11 23:10:18 +07:00
64287560e4 [pandoratv] Add support for new URL format (closes #15131) 2018-01-11 23:06:56 +07:00
37941fe204 [ChangeLog] Update after #14687
[ci skip]
2018-01-11 20:36:06 +08:00
a90641fe87 [ximalaya_extractor] Add new extractor ximalaya (#14687)
* [ximalaya_extractor] Add new extractor

* format change according by flake8

* changes accoring to review by @yan12125 at github pull #14687

* change %d to %s in a temp str

* seond changes accoring to review by @yan12125 at github pull #1468

* improve TESTS about contains

* changes accoring to third review by @yan12125 at github pull #1468

* forth changes accoring to forth review by @yan12125 at github pull #1468
2018-01-11 20:35:09 +08:00
1b79daffd9 [digg] Improve extraction 2018-01-10 22:19:51 +07:00
e654829b4c [digg] Add extractor (closes #15214) 2018-01-10 21:24:22 +07:00
2b4e1ace4a [limelight] Tolerate empty pc formats (closes #15150, closes #15151, closes #15207) 2018-01-10 05:39:57 +07:00
310ea4661d [ndr:embed:base] Make separate formats extraction non fatal (closes #15203) 2018-01-09 22:04:50 +07:00
5b23845125 Credit @sprhawk for the Weibo extractor (#15079) 2018-01-09 19:35:39 +08:00
0f71de0761 [ChangeLog] Update after #15079 2018-01-09 18:13:49 +08:00
4df1098c3f Merge branch 'sprhawk-weibo' 2018-01-09 18:13:11 +08:00
5eca00a2e3 [weibo] Misc improvements 2018-01-09 18:12:55 +08:00
1dd38dc0f4 Merge branch 'weibo' of https://github.com/sprhawk/youtube-dl into sprhawk-weibo 2018-01-09 17:31:52 +08:00
8005dc68cb [ok] Add support for live streams 2018-01-08 21:53:03 +07:00
a39e15c516 [canalplus] fix extraction(closes #15072) 2018-01-07 22:15:44 +01:00
7643916a37 [ChangeLog] update after #15188
[ci skip]
2018-01-08 01:32:13 +08:00
3a513f29ad fix bilibili extraction (closes #15171) 2018-01-08 01:30:04 +08:00
950b5f2969 release 2018.01.07 2018-01-07 23:52:16 +07:00
8faa9576bb [ChangeLog] Actualize 2018-01-07 23:48:56 +07:00
b0ead0e09a [jwplatform] Add support for multiple embeds (closes #15192) 2018-01-07 21:49:23 +07:00
0a5b1295b7 [motherless:group] Relax entry extraction and add a fallback scenario 2018-01-07 00:31:53 +07:00
a133eb7764 [motherless:group] Capture leading slash of video path 2018-01-07 00:02:41 +07:00
f12628f934 [mitele] Fix extraction (closes #15186) 2018-01-06 23:58:00 +07:00
45283afdec [motherless] Add support for groups 2018-01-06 23:33:40 +07:00
b7c74c0403 [lynda] Relax _VALID_URL (closes #15185) 2018-01-06 23:12:30 +07:00
0b0870f9d0 [soundcloud] Fallback to avatar picture for thumbnail (closes #12878) 2018-01-05 08:25:42 +07:00
c2f18e1c49 [ChangeLog] Update after #15137
[skip ci]
2018-01-04 22:28:00 +08:00
da35331c6c [youku] Fix list extraction.(close #15135) (#15137)
* [youku] Fix list extraction.(close #15135)

Change-Id: I2e9c920143f4f16012252625943a8f18b8ff40eb

* [youku] Remove KeyError try-except

Change-Id: Ic46327905cbef1356b7b12d5eb3db5d9746ca338
2018-01-04 22:25:28 +08:00
de329f64ab [openload] Fix extraction (closes #15166) 2018-01-04 13:26:08 +08:00
75ba0efb52 [lynda] Skip invalid subtitles (closes #15159) 2018-01-03 16:41:28 +07:00
f0c6c2bce2 [twitch] Pass video id to url_result when extracting playlist 2018-01-03 16:22:55 +07:00
9650c3e91d [rtve.es:alacarta] Fix extraction of some new URLs 2018-01-02 21:12:39 +01:00
b5e531f31a [acast] Fix extraction 2018-01-02 23:32:17 +07:00
7a6c204fcb [travis] Add Jython build 2018-01-02 21:13:41 +07:00
d7cd9a9e84 [utils] Fix youtube-dl under PyPy3 on Windows 2018-01-01 22:48:27 +07:00
54009c246e [travis] Add PyPy builds 2018-01-01 21:54:28 +07:00
b300cda476 [YoutubeDL] Output python implementation in debug header 2018-01-01 21:52:24 +07:00
6648fd8ad6 changed to use .get to get field from json object 2018-01-01 18:33:14 +08:00
04cf1a191a release 2017.12.31 2017-12-31 04:30:49 +07:00
c95c08a856 [ChangeLog] Actualize 2017-12-31 04:28:01 +07:00
126f225bcf [extractor/common] Add container meta field for formats extracted in _parse_mpd_formats 2017-12-31 04:04:09 +07:00
4f5cf31977 [slutload] Add support for mobile URLs 2017-12-31 01:41:07 +07:00
77341dae14 [abc:iview] Improve extraction and bypass geo restriction (closes #14782) 2017-12-31 01:27:28 +07:00
2e65e7db9e [abc:iview] Fix extraction (closes #14711)
ABC dropped unmetering, so change to metered hls urls which
require auth.
2017-12-31 01:27:22 +07:00
538d4f8681 [downloader/hls] Use HTTP headers for key request 2017-12-31 01:15:35 +07:00
620ee8712e [openload] Fix extraction (closes #15118) 2017-12-30 15:03:13 +08:00
2ca7ed41fe [mediasite] Improve extraction and code style, add support for DASH (closes #11185, closes #14343, refs #5428) 2017-12-30 08:04:43 +07:00
8056c8542d [mediasite] Add extractor, subsume sandia and collegerama extractors 2017-12-30 07:23:41 +07:00
2501d41ef4 [common] use AACL as the default fourcc when AudioTag is 255 2017-12-30 07:22:07 +07:00
d97cb84b31 [ufctv] Add new extractor(closes #14520) 2017-12-30 00:30:41 +01:00
2c8e11b4af [pluralsight] Fix missing first line of subtitles (closes #11118) 2017-12-30 05:56:47 +07:00
d2c5b5a951 [openload] Fallback on f-page extraction (closes #14665, closes #14879) 2017-12-30 05:53:56 +07:00
580f3c79d5 [vimeo] Improve password protected videos extraction (closes #15114) 2017-12-30 03:54:14 +07:00
48058d82dc replace unused _download_webpage_handle with _download_webpage 2017-12-30 01:14:21 +08:00
6a41a12d29 replace split with strip_jsonp 2017-12-30 01:11:30 +08:00
5c97ec5ff5 replace urlencode.encode with urlencode_postdata 2017-12-30 01:08:56 +08:00
9d6ac71c27 [extractor/common] Fix extraction of DASH formats with the same representation id (closes #15111) 2017-12-29 23:14:56 +07:00
84f085d4bd [aws] fix canonical/signed headers generation in python 2(closes #15102) 2017-12-29 00:13:40 +01:00
a491fd0c6f release 2017.12.28 2017-12-28 23:12:56 +07:00
99277daaac [ChangeLog] Actualize 2017-12-28 23:10:42 +07:00
640788f6f4 [internazionale] Improve extraction (closes #14973) 2017-12-27 23:27:48 +07:00
1ae0f0a21d [internazionale] Add extractor 2017-12-27 23:27:43 +07:00
616bb95b28 [playtvak] Relax video regex and make description optional 2017-12-27 22:57:26 +07:00
be069839b4 [filmweb] improve extraction 2017-12-26 19:41:08 +01:00
c33de004e1 Merge branch 'master' of github.com:rg3/youtube-dl into weibo 2017-12-26 22:27:26 +08:00
42a1012c77 fix according to "https://github.com/rg3/youtube-dl/pull/15079#discussion_r158688607" 2017-12-26 22:26:01 +08:00
a14001a5a1 [Filmweb] Add extractor 2017-12-26 15:19:37 +01:00
db145ee54a [espn] Add new extractor for http://fivethirtyeight.com(closes #6864) 2017-12-26 14:20:21 +01:00
45d20488f1 [umg:de] Add new extractor(closes #11582)(closes #11584) 2017-12-26 12:32:04 +01:00
2593651224 fix compat_urllib_request for python2.7 2017-12-26 16:46:01 +08:00
951043724f re-format code to pass flake8 2017-12-26 16:38:51 +08:00
d2be5bb5af change to use compat urllib 2017-12-26 16:28:47 +08:00
447a5a710d added weibo mobile site support 2017-12-26 16:24:56 +08:00
0f897e0929 [espn] add support for espnfc and extract more formats(closes #8053) 2017-12-25 23:29:09 +01:00
173558ce96 [ChangeLog] Update after #15065 2017-12-25 22:06:18 +08:00
d3ca283235 [youku] Add test case.
Some playlist has no data-id value.

Change-Id: I97455f2907f08bda03b538cdc13ec827e2f8ce26
2017-12-25 22:02:47 +08:00
d99a1000c7 [youku] Fix list extraction.(close #15065)
Change-Id: I578fdc5b69509bdcd8d3191e3917afe47c234ff6
2017-12-25 22:02:47 +08:00
a75419586b [openload] Remove a confusing exception
If phantomjs is not installed, there's an error besides the missing
phantomjs exception:

Exception ignored in: <bound method PhantomJSwrapper.__del__ of <youtube_dl.extractor.openload.PhantomJSwrapper object at 0x7f8ad5e78278>>
Traceback (most recent call last):
  File "/home/yen/Projects/youtube-dl/youtube_dl/extractor/openload.py", line 142, in __del__
    os.remove(self._TMP_FILES[name].name)
AttributeError: 'PhantomJSwrapper' object has no attribute '_TMP_FILES'
2017-12-24 20:47:42 +08:00
273c23d960 [openload] Add support for oload.stream (closes #15070) 2017-12-24 13:53:27 +07:00
b954e72c87 [ChangeLog] typo 2017-12-23 23:42:02 +08:00
116561697d [ChangeLog] Update after #14903 2017-12-23 23:41:24 +08:00
0e25a1a278 [youku] Update ccode
Change-Id: Id397e814e81ff560506d68563b7409eebbe5943d
2017-12-23 23:34:42 +08:00
307a7588b0 release 2017.12.23 2017-12-23 21:24:18 +07:00
c2f2f8b120 [kaltura] Fix typo 2017-12-23 21:22:41 +07:00
f5a6321107 [ChangeLog] Actualize 2017-12-23 21:17:53 +07:00
69d69da98a [kaltura] Add another embed pattern for entry_id
For cases when player configuration map is setup via indexing operator, e.g. kalturaPlayerConfiguration_1_lre6rg3i_10[entry_id] = 1_lre6rg3i (see https://www.heise.de/video/artikel/odcast-c-t-uplink-20-1-Apple-CarPlay-vs-Android-Auto-Galileo-3D-Sound-erklaert-3919694.html)
2017-12-23 21:17:53 +07:00
5c5e60cff8 [voot] Fix video identification 2017-12-23 21:17:53 +07:00
2132edaa03 [extractor/common] Move X-Forwarded-For setup code into _request_webpage 2017-12-23 21:17:53 +07:00
4b7dd1705a [7plus] Add new extractor(closes #15043) 2017-12-23 13:22:20 +01:00
9e3682d555 [MANIFEST.in] Include all test data in PyPI package 2017-12-22 23:53:27 +07:00
3e191da6d9 [Makefile] Add AUTHORS to youtube-dl.tar.gz 2017-12-22 23:46:08 +07:00
963d237d26 Add LICENSE, AUTHORS and ChangeLog to PyPI package (closes #15054) 2017-12-22 23:38:16 +07:00
d2d766bc6d [animeondemand] Fix typo 2017-12-20 23:18:14 +07:00
17c3aced5d [animeondemand] Relax login error regex 2017-12-19 22:53:04 +07:00
78466fcab5 [shahid] add support for show pages(closes #7401) 2017-12-19 02:00:38 +01:00
3961c6cb9d [YoutubeDL] Add support for playlist_uploader and playlist_uploader_id in output template (closes #11427, #15018) 2017-12-19 03:53:44 +07:00
07aeced68e [youtube] Extract uploader, uploader_id and uploader_url for playlists (#11427, #15018) 2017-12-19 03:51:28 +07:00
c10c93238e [extractor/common] Introduce uploader, uploader_id and uploader_url meta fields for playlists (#11427, #15018) 2017-12-19 03:51:03 +07:00
4a109f81bc [afreecatv] Improve format extraction (closes #15019) 2017-12-19 00:38:39 +07:00
99081da90c [downloader/fragment] Encode filename of fragment being removed (closes #15020) 2017-12-18 03:31:53 +07:00
7e81010987 [cspan] add support for audio only pages and catch page errors(closes #14995) 2017-12-17 19:15:59 +01:00
549bb416f5 [mailru] Fix issues and improve (closes #14904) 2017-12-17 18:38:27 +07:00
25475dfab3 [mailru] Add support for embed URLs 2017-12-17 18:37:03 +07:00
3dfa9ec213 [crunchyroll] Future-proof XML element checks(closes #15013) 2017-12-17 09:15:44 +01:00
06dbcd7be4 [cbslocal] Fix timestamp extraction (closes #14999, closes #15000) 2017-12-16 21:57:30 +07:00
b555ae9bf1 [utils] Add another date format pattern (#14999) 2017-12-16 21:56:16 +07:00
c402e7f3a0 [discoverygo] correct ttml subtitle extension 2017-12-16 12:55:44 +01:00
498a8a4ca5 [vk] Make view count optional (closes #14979) 2017-12-15 22:53:56 +07:00
d05ba4b89e [disney] skip Apple FairPlay formats(#14982) 2017-12-15 09:28:07 +01:00
23f511f5c7 [voot] sort formats 2017-12-15 09:05:59 +01:00
1c4804ef9b [voot] fix format extraction(closes #14758) 2017-12-14 23:05:43 +01:00
8ff2b16435 release 2017.12.14 2017-12-14 05:19:21 +07:00
c6a5a811a1 [ChangeLog] Actualize 2017-12-14 05:15:51 +07:00
3fae11ac00 [itv] Improve extraction, extract more subtitles and duration (closes #14944) 2017-12-14 04:49:07 +07:00
7974e289a1 [postprocessor/xattr] Clarify NO_SPACE message (#14970) 2017-12-14 01:05:02 +07:00
6bf9c28b0a [byutv] add support for geo restricted videos 2017-12-13 17:51:56 +01:00
bec49996c6 [downloader/http] Return actual download result (closes #14971) 2017-12-13 23:49:05 +07:00
c8be7d5f74 [byutv] Fix extraction (closes #14966, closes #14967) 2017-12-13 23:14:30 +07:00
15960255fe [tbs] fix typo 2017-12-12 18:16:45 +01:00
6b2d8c9182 [bbc.co.uk] fix extraction for 320k m3u8 streams
broken since 197224b7a4
2017-12-12 18:04:05 +01:00
e6b8803d59 [toutv] add support special video urls(closes #14179) 2017-12-12 11:11:44 +01:00
cb0c2310fb [discovery] fix free videos extraction(#14157)(#14954) 2017-12-12 10:33:03 +01:00
23b6e23002 [tvnow] fix extraction(closes #7831) 2017-12-11 21:36:12 +01:00
127e98d31d [nickelodeon:br] correct extractor name 2017-12-11 17:21:31 +01:00
e4f201bc1b [extractors] add import for NickBrIE 2017-12-11 17:19:16 +01:00
08d77a95c9 [nickelodeon:br] add support for Nickelodeon(Brazil) websites(closes #14893) 2017-12-11 17:18:05 +01:00
5868079e99 [nick.com] improve extraction(fixes #14876) 2017-12-11 14:37:03 +01:00
b6f78d76c1 [tbs] fix extraction(fixes #13658) 2017-12-11 13:39:25 +01:00
0c69958844 add other properties; remove print verbose 2017-12-11 16:02:14 +08:00
3281af3464 a working version 2017-12-11 15:56:54 +08:00
29ac31afaf simply get the correct webpage, but not parsed to extract information 2017-12-11 12:26:19 +08:00
1fa0dce2c0 release 2017.12.10 2017-12-10 23:18:53 +07:00
fa1dd6d2cd [ChangeLog] Actualize 2017-12-10 23:15:24 +07:00
c38970ca10 [culturebox] Improve video id extraction (closes #14947) 2017-12-10 22:46:21 +07:00
51f2863357 [twitter] improve extraction(closes #14197) 2017-12-10 14:11:09 +01:00
913b61eeee [udemy] Extract more HLS formats 2017-12-09 20:02:54 +07:00
6f1ec339a0 [udemy] Improve course id extraction (closes #14938) 2017-12-09 20:02:49 +07:00
a3de5e6c0e [stretchinternet] Fix issues and improve (closes #14576) 2017-12-09 17:59:08 +07:00
f4cc03d60b [stretchinternet] Add extractor 2017-12-09 17:58:49 +07:00
2a57b62b80 [ellentube] Fix issues, improve and simplify (closes #14570) 2017-12-09 02:16:54 +07:00
e2707a832c [ellentube] Fix extraction (closes #14407) 2017-12-09 02:16:48 +07:00
1115271ac6 [raiplay:playlist] Fix issues and improve (closes #14563) 2017-12-09 00:48:04 +07:00
d21d0ba6c1 [raiplay:playlist] Add extractor 2017-12-09 00:47:40 +07:00
a670b1ba26 [README.md] Add is_live, start_time and end_time to output template section (closes #14926) 2017-12-07 22:16:41 +07:00
1bd4fc96e6 [sonyliv] extract higher quality formats and bypass geo restriction(closes #14922) 2017-12-07 08:46:30 +01:00
684ae10236 [fox] add support for adobe pass auth and extract subtitles(close #14489)(closes #14205) 2017-12-06 22:56:14 +01:00
3c4fbfeca2 [dailymotion] remove dailymotion cloud extractor(closes #6794)
https://web.archive.org/web/20160312110217/https://www.dmcloud.net/
2017-12-06 10:56:48 +01:00
b271e33526 [xhamster] Add support for mobile URLs and fix thumbnail extraction 2017-12-06 00:08:31 +07:00
d3f8b76b69 [extractor/generic] Fix typo (closes #14902)
Don't pass video_id as mpd_id
2017-12-05 23:11:15 +07:00
91328f26b0 [ard] Skip invalid stream URLs (closes #14906) 2017-12-05 23:01:57 +07:00
61d18c8a4b [porncom] Fix metadata extraction (closes #14911) 2017-12-05 22:42:02 +07:00
c94427dd60 [pluralsight] Detect agreement request (#14913) 2017-12-05 22:34:56 +07:00
d4f05d4731 [utils] add sami mimetype to mimetype2ext 2017-12-03 00:04:43 +01:00
d7df308981 [toutv] fix login(closes 14614) 2017-12-02 20:22:40 +01:00
0d56eddc59 release 2017.12.02 2017-12-02 21:34:34 +07:00
e25ee72657 [ChangeLog] Actualize 2017-12-02 21:29:06 +07:00
78593e294c Add references for #14844 2017-12-02 21:22:43 +07:00
593f2f7989 [downloader/fragment] Commit part file after each fragment
In order to obtain correct resume_len on next iteration
2017-12-02 21:21:11 +07:00
603fc4e0ea [extractor/common] Add durations for DASH fragments with bare SegmentURLs 2017-12-02 21:21:01 +07:00
41bf647e89 [extractor/common] Add support for DASH manifests with SegmentLists with bare SegmentURLs 2017-12-02 21:16:36 +07:00
fea92aa65d [xhamster] Fix extraction (closes #14884) 2017-12-02 19:04:59 +07:00
0981585bef [youku] Update ccode (closes #14872) 2017-12-02 18:16:22 +07:00
f5ac68d88f [mnet] fix format extraction(fixes #14883) 2017-11-30 23:45:33 +01:00
1663b32946 [xiami] add Referer header to api request 2017-11-29 20:36:55 +01:00
5ea765fb72 [mtv] correct scc extention in extracted subtitles(closes #13730) 2017-11-29 17:50:38 +01:00
fb61b57d0f [vvvvid] fix extraction for kenc videos(fixes #13406) 2017-11-29 16:09:45 +01:00
07cf18b9c5 [br] add support for BR Mediathek videos(fixes #14560)(fixes #14788) 2017-11-29 14:21:38 +01:00
5f699251e9 [daisuki] add support for motto.daisuki.com(fixes #14681) 2017-11-28 10:57:22 +01:00
a3474aa59e [Odnoklassniki] fix api metadata request(fixes #14862) 2017-11-28 09:04:51 +01:00
115afb77ec [itv] update hls formats extraction 2017-11-27 21:59:27 +01:00
53f024e7c5 [pbs] add another media id regex 2017-11-27 16:55:27 +01:00
ffe6979ef9 [utils] add hvc1 codec code to parse_codecs 2017-11-27 16:55:27 +01:00
dafb4c6647 [Makefile] Include setup.cfg in the tarball (closes #14857) 2017-11-27 22:49:35 +08:00
82a62de192 [Makefile,devscripts/run_tests.sh] Actually exclude network tests
Closes #14858
2017-11-27 21:32:06 +08:00
f58a506044 [test_InfoExtractor] Fix flake8 2017-11-27 21:30:47 +08:00
5ddeb7702a release 2017.11.26 2017-11-26 21:49:22 +07:00
6c07f0b288 [ChangeLog] Actualize 2017-11-26 21:37:27 +07:00
e94d1adc36 Add testdata to youtube-dl.tar.gz (closes #14854) 2017-11-26 21:10:32 +07:00
d08dcd2dbd [test_YoutubeDL] Fix typo (closes #14856) 2017-11-26 21:06:14 +07:00
7512aa986f Fix some only matching tests (closes #14855) 2017-11-26 20:53:10 +07:00
93f3f10cdc [fczenit] fix extraction 2017-11-25 19:28:26 +01:00
87dac57cf6 [firstpost] remove extractor 2017-11-25 18:50:15 +01:00
b485d5d6bf [nexx] make http format ids more consistent 2017-11-25 18:36:31 +01:00
a238a868ba [freespeech] fix extraction 2017-11-25 18:25:00 +01:00
c0f647a179 [nexx] extract more formats 2017-11-25 18:13:26 +01:00
6ff27b8d5a [openload] Don't use bare except when removing temp files 2017-11-26 00:05:28 +07:00
9ef909f2b2 [openload] Add support for openload.link 2017-11-26 00:04:13 +07:00
8cfbcfab9a [tnaflix] Extract common parts of tnaflix and empflix 2017-11-25 23:42:20 +07:00
b7785cf156 [empflix] Relax _VALID_URL 2017-11-25 23:42:20 +07:00
9105523818 [empflix] Fix extractrion 2017-11-25 23:42:20 +07:00
dbb25af657 [tnaflix] Don't modify download URLs (closes #14811) 2017-11-25 23:42:20 +07:00
fe4bfe36e1 [gamersyde] remove extractor 2017-11-25 15:58:28 +01:00
6f5c598a28 [france2.fr:generation-what] fix extraction 2017-11-25 15:49:49 +01:00
cd9ff4ec5b [massengeschmacktv] Add support for Massengeschmack TV(replaces Fernseh Kritik TV) 2017-11-24 20:00:01 +01:00
c6c6a64aa5 [fox9] fix extraction 2017-11-24 19:00:56 +01:00
e0a8686f48 [faz] fix extraction and add support for Perform Group embeds(fixes #14714) 2017-11-24 18:42:41 +01:00
6049176471 [ChangeLog] Update after #14828
[skip ci]
2017-11-24 21:40:51 +08:00
805f5bf759 [Generic] ie_key in JWPlatform test 72 2017-11-24 21:39:55 +08:00
32ad4f3faf [JWPlatform] Use non-capturing group in RE
Per @yan12125.
2017-11-24 21:39:55 +08:00
6899b1d9e8 [Generic] Update test 69 (suffolk/sjc)
suffolk.edu/sjc => suffolk.edu/sjc/live.php

Unfortunately it only transmits video a few mornings per month, so
leaving the 'skip' is probably appropriate. Updating the 'skip- to
include the calendar information though.
2017-11-24 21:39:55 +08:00
939be9adfe [JWPlatform] Support iframes
Support content.jwplatform... src attributes inside <iframe> tags in
addition to <script> tags. Just a regexp change.

Add a test (currently Generic_72).
2017-11-24 21:39:55 +08:00
2688664762 [culturebox] Fix extraction (closes #14827) 2017-11-23 06:39:11 +07:00
8f63941104 [youku] Fix extraction; update ccode (closes #14815) 2017-11-22 22:49:48 +08:00
a9efdf3d4a [livestream] make smil extraction non fatal(fixes #14792) 2017-11-19 12:59:31 +01:00
f610dbb05f [extractor/common] Use final URL when dumping request (closes #14769) 2017-11-18 19:04:56 +07:00
38db52adf3 [drtuber] Add support for mobile URLs 2017-11-17 01:50:07 +07:00
3192d4bc7a [spankbang] Add support for mobile URLs and fix test 2017-11-17 01:05:04 +07:00
9cbd4dda10 [instagram] Fix description, timestamp and counters extraction (closes #14755) 2017-11-15 22:14:54 +07:00
08e45b39e7 release 2017.11.15 2017-11-15 00:15:42 +07:00
fae0eb42ec [ChangeLog] Actualize 2017-11-15 00:02:54 +07:00
ea2295842f [common] skip Apple FairPlay m3u8 manifests(closes #14741) 2017-11-14 17:41:30 +01:00
a2b6aba8de [vshare] Improve extraction, fix formats sorting and carry long lines 2017-11-14 22:50:15 +07:00
ff31f2d5c3 [vshare] Capture and output error message 2017-11-14 22:39:54 +07:00
0987f2ddb2 [vshare] Fix extraction (closes #14473) 2017-11-14 22:34:45 +07:00
5871ebac47 [YoutubeDL] Fix playlist range optimization for --playlist-items (closes #14740) 2017-11-14 01:43:20 +07:00
05dee6c520 [crunchyroll] extract old rtmp formats 2017-11-13 19:15:49 +01:00
27adc9ec65 [tva] fix extraction(closes #14736) 2017-11-13 11:24:15 +01:00
388beb86e0 [gamespot] add test for #14652 2017-11-13 10:30:12 +01:00
d4e31b72b9 [gamespot] lower the preference of http formats(#14652) 2017-11-13 10:24:35 +01:00
5fc12b9549 [instagram:user] Fix extraction (closes #14699) 2017-11-12 18:36:18 +07:00
af85ce29c6 [ccma] Fix typo 2017-11-12 13:25:21 +07:00
e4d9586562 Remove sensitive data from logging in messages 2017-11-11 20:52:12 +07:00
79d1f8ed68 [gamespot] add support for article URLS(closes #14652) 2017-11-11 13:03:16 +01:00
a5203935d6 [gamespot] skip Brightcove Once http formats(#14652) 2017-11-11 13:03:16 +01:00
59d2e6d04f [cartoonnetwork] Update tokenizer_src (closes #14666) 2017-11-11 04:59:48 +07:00
a9543e37c8 [wsj] Recognize another URL pattern (closes #14704) 2017-11-11 00:29:08 +08:00
61fb07e156 [pandatv] Modernize (closes #14693) 2017-11-09 23:30:25 +07:00
4222346fb2 [pandatv] Update API URL and sign format URLs 2017-11-09 23:26:46 +07:00
cc6a960e13 use older login method(closes #11572) 2017-11-08 20:30:05 +01:00
f34b841b51 release 2017.11.06 2017-11-06 22:39:24 +07:00
e0998333fa [ChangeLog] Actualize 2017-11-06 22:36:46 +07:00
909191de91 [hotstar:playlist] Fix issues and improve (closes #12465) 2017-11-05 19:15:40 +07:00
477c97f86b [hotstar:playlist] Add extractor 2017-11-05 19:15:34 +07:00
6e71bbf4ab [hotstar] Bypass geo restriction (closes #14672) 2017-11-05 16:12:56 +07:00
181e381fda [test_InfoExtractor] Add test for #14660 2017-11-04 22:15:58 +07:00
187ee66c94 [extractor/common] Add protocol for f4m formats 2017-11-04 22:11:39 +07:00
48107c198b [f4m] Prefer baseURL for relative URLs (closes #14660) 2017-11-04 22:10:55 +07:00
cd670befc4 [22tracks] Remove extractor (closes #11024) 2017-11-02 23:48:43 +07:00
44cca168cc [skysport] add support ooyala embed_token protected videos(fixes #14641) 2017-11-02 14:16:15 +01:00
b0f4331002 [gamespot] extract formats referenced with new data fields(#14652) 2017-11-02 13:30:50 +01:00
044eeb1455 [extractor/common] Respect URL query in _extract_wowza_formats (closes #14645) 2017-11-01 23:39:26 +07:00
8fe767e072 [spankbang] Detect unavailable videos (closes #14644) 2017-10-31 23:05:25 +07:00
6d0630d880 release 2017.10.29 2017-10-29 07:22:53 +07:00
518d357b46 [ChangeLog] Actualize 2017-10-29 07:21:33 +07:00
514e8aefd4 [egghead] Fix extraction (closes #14388) 2017-10-29 07:11:37 +07:00
9211e3319e [extractor/common] Prefix format id for audio only HLS formats 2017-10-29 07:05:55 +07:00
056653bbb1 [utils] Add support for zero years and months in parse_duration 2017-10-29 07:04:48 +07:00
c3206d02e9 [fxnetworks] Extract series metadata 2017-10-29 05:20:18 +07:00
eb4b5818e2 [younow] Fix issues and improve extraction (closes #9255, closes #9432, closes #12436) 2017-10-29 04:18:43 +07:00
47a8587915 [younow] Add extractor 2017-10-29 04:17:03 +07:00
8e01f3ca81 [dctptv] Fix extraction (closes #14599) 2017-10-28 22:58:01 +07:00
f2332f18e6 [youtube] Restrict embed regex (#14600) 2017-10-27 22:26:43 +07:00
7c1f419341 [vimeo] Restrict iframe embed regex (closes #14600) 2017-10-27 22:21:47 +07:00
30e6161799 [soundgasm] Improve extraction (closes #14588) 2017-10-26 23:16:16 +07:00
dc24a7d4a2 [myvideo] Remove extractor (closes #8557)
Redirects to store.maxdome.de
2017-10-25 23:27:55 +07:00
d673ab6562 [nbc] Add support for classic-tv videos 2017-10-25 23:23:27 +07:00
b8c6ffc518 [vrtnu] Add support for cookies authentication and simplify (#11873) 2017-10-25 23:21:51 +07:00
7913e0fca7 [canvas] Add support for vrt.be/vrtnu (closes #11873) 2017-10-25 23:17:28 +07:00
cdd1ce92c4 [twitch:clips] Fix title extraction 2017-10-23 23:12:50 +07:00
55c727a547 [ndtv] Add support for sub-sites 2017-10-22 08:32:20 +07:00
36e2d3ca43 [dramafever] Fix login error message extraction 2017-10-22 08:16:30 +07:00
f7a5038305 [travis] Disable IRC notifications 2017-10-22 02:46:28 +07:00
9ff6273cae [nickru] Add support for more sites 2017-10-22 01:51:01 +07:00
f03ee0b372 [nickde] Add support for nickelodeon.be 2017-10-22 01:42:44 +07:00
cf6bda312b [nickde] Add support for nick.ch 2017-10-22 01:30:35 +07:00
3ebbd9991e [nick] Add support for more nickelodeon sites (closes #14553) 2017-10-22 01:26:58 +07:00
21ce434051 [travis] Enable IRC notifications
Let's see how is it verbose now
2017-10-21 02:14:25 +07:00
5c0e5bc4df [README.md] Add build status bagde 2017-10-21 02:11:11 +07:00
9a9de2d7b2 [travis] Allow download tests to fail and fast finish 2017-10-21 01:58:45 +07:00
424505df76 [azmedien] Fix test 2017-10-21 01:10:56 +07:00
fa3f0fd856 release 2017.10.20 2017-10-20 23:40:25 +07:00
c9dcd4b0c5 [ChangeLog] Actualize 2017-10-20 23:37:55 +07:00
fc5c47d13c [parliamentliveuk] Fix extraction (closes #14524) 2017-10-20 23:31:13 +07:00
a26a3c6d34 [soundcloud] Update client id (closes #14546) 2017-10-20 21:43:34 +07:00
382fa456ea [ChangeLog] Fix typo 2017-10-19 23:36:32 +07:00
e1d168e592 [servus] Add extractor (closes #14362) 2017-10-19 22:17:20 +07:00
ca1c9f26fa [unity] Add extractor (fixes #14528) 2017-10-19 04:46:06 +07:00
6f3b4a98c9 [downloader/fragment] Report warning instead of error on inconsistent download state 2017-10-17 22:53:34 +07:00
fa4bc6e712 [youtube] replace youtube redirect urls in description(fixes #14517) 2017-10-17 10:07:37 +00:00
6b9cbd023f [pbs] restrict direct video url regex(fixes #14519) 2017-10-17 09:23:11 +00:00
c233003afe [megaphone] Fix deprecated escape sequence 2017-10-17 15:39:06 +08:00
83fcf19e2d [drtv] Respect preference for direct http formats (#14509) 2017-10-16 05:48:45 +07:00
acc4ea6237 [eporner] Add support for embed URLs (closes #14507) 2017-10-16 05:11:25 +07:00
8cc1840ccb [arte] Capture and output error message 2017-10-15 22:12:34 +07:00
a9ee4f6e49 [downloader/hls] Fix total fragments count when ad fragments exist 2017-10-15 11:03:54 +07:00
aaab8c5e71 [niconico] Improve uploader metadata extraction robustness (closes #14135) 2017-10-15 10:40:57 +07:00
7e721e35da release 2017.10.15.1 2017-10-15 06:16:41 +07:00
bd7e1406b3 [ChangeLog] Actualize 2017-10-15 06:15:37 +07:00
74c42d9ec3 [downloader/hls] Ignore anvato ad fragments (closes #14496) 2017-10-15 06:13:48 +07:00
5efaf43c93 [downloader/fragment] Output ad fragment count 2017-10-15 06:13:07 +07:00
4827270526 [scrippsnetworks:watch] Bypass geo restriction 2017-10-15 06:11:35 +07:00
ee093a0ea0 [anvato] Add ability to bypass geo restriction 2017-10-15 06:11:02 +07:00
9bb2c7673e [redditr] Fix extraction for URLs with query (closes #14495) 2017-10-15 03:38:34 +07:00
715534083d release 2017.10.15 2017-10-15 02:26:58 +07:00
ee88c1cbc6 [ChangeLog] Actualize 2017-10-15 02:26:10 +07:00
57eb45b111 [scrippsnetworks:watch] Add support for geniuskitchen.com 2017-10-15 02:01:16 +07:00
b21ab85088 [scrippsnetworks:watch] Fix extraction (closes #14389) 2017-10-15 01:57:43 +07:00
210a2720bc [anvato] Process master m3u8 manifests
>>> Individual m3u8 manifests are not always present, e.g. anvato:anvato_scripps_app_web_prod_0837996dbe373629133857ae9eb72e740424d80a:4173834
2017-10-15 01:44:57 +07:00
685e87b61f [youtube] Fix relative URLs in description 2017-10-14 20:26:52 +07:00
c9bd503e7d [spike] bypass geo restriction 2017-10-13 08:41:57 +00:00
94a530c6cb [howstuffworks] add support for more domains 2017-10-12 19:03:47 +00:00
e650659b94 [infoq] fix http format downloading 2017-10-12 17:39:51 +00:00
2637fadc38 [generic] fix some of the tests 2017-10-12 16:14:43 +00:00
50d808f5c9 [common] add support for jwplayer youtube embeds 2017-10-12 16:12:47 +00:00
7a64c33aee [rtlnl] add support for another type of embeds 2017-10-12 16:09:06 +00:00
b0def2c297 [onionstudios] add support for bulbs-video embeds 2017-10-12 16:05:25 +00:00
81ce479f4d [udn] fix extraction 2017-10-12 16:04:41 +00:00
414e709405 [shahid] fix extraction(fixes #14448) 2017-10-12 09:20:39 +00:00
645ed3e7c9 [ChangeLog] Update after #14471
[skip ci]
2017-10-12 12:12:37 +08:00
c0bddd6d65 [kaltura] Ignore Widevine encrypted video (.wvm)
There is currently no public method to decrypt this, and there may be
other streams available that can be downloaded.

Example URL, has `.wvm` and `.mp4` formats:
https://www.voot.com/shows/bigg-boss-s11/11/538936/bigg-boss-extra-dose-arshi-s-quirky-demand/541700
2017-10-12 12:09:58 +08:00
1baba7f4a8 [vh1] Adding coding cookie 2017-10-12 12:02:26 +08:00
344d1a6794 [vh1] fix extraction(fixes #9613) 2017-10-11 20:52:14 +00:00
76581082f6 release 2017.10.12 2017-10-12 01:06:28 +07:00
2f0eb0a68a [ChangeLog] Actualize 2017-10-12 01:05:14 +07:00
7fee3377dc [steam] fix extraction(fixes #14067) 2017-10-11 17:50:08 +00:00
ff3f1a62f0 [funk] Add extractor (closes #14464) 2017-10-12 00:44:13 +07:00
694b61545c [nexx] Add support for shortcuts and relax domain id extraction 2017-10-12 00:41:20 +07:00
af0f74288d [YoutubeDL] Improve _default_format_spec (closes #14461) 2017-10-11 23:48:05 +07:00
9e38dbb19c [voxmedia] add support for recode.net(fixes #14173) 2017-10-11 15:50:20 +00:00
782195a9d4 [once] add support for vmap urls 2017-10-11 15:50:20 +00:00
26bae2d965 [generic] Add support for channel9 embeds (closes #14469) 2017-10-11 21:59:30 +07:00
5fe75f976f [tva] fix extraction(fixes #14328) 2017-10-11 14:15:52 +00:00
4fe4bda287 [tubitv] add support for new url format(fixes #14460) 2017-10-11 11:36:05 +00:00
cdab1df912 [afreecatv] remove AfreecaTVGlobalIE
the website now show this message
> Global AfreecaTV will be merged and integrated on July 20th, 2017.
Every user around the world are now able to interact with one another on
www.afreecatv.com!
2017-10-11 10:04:46 +00:00
dfc80bdd2e [ChangeLog] Update after #14420 2017-10-11 02:03:00 +08:00
04af3aca04 Remove YoutubeSharedVideoIE https://github.com/rg3/youtube-dl/issues/14303 2017-10-11 02:01:18 +08:00
d0f2d64114 [slideslive] Add extractor (closes #2680) 2017-10-10 23:45:10 +07:00
01c742ecd0 [facebook] Support thumbnails (closes #14416) 2017-10-10 23:20:38 +08:00
9e71f88105 [vvvvid] Fix typo 2017-10-10 03:48:26 +07:00
ae5af89079 [hrti:playlist] Relax _VALID_URL 2017-10-09 23:52:39 +07:00
197224b7a4 Fix some regexes 2017-10-09 23:50:53 +07:00
8992331621 [wdr] Relax media link regex (closes #14447) 2017-10-08 21:36:50 +07:00
b0dde6686c [hrti] Relax _VALID_URL 2017-10-08 05:40:08 +07:00
a22ccac1f0 [fox] Delegate to uplynk:preplay (#14147) 2017-10-08 01:34:17 +07:00
8b561bfc9d [youtube] Add support for hooktube.com (closes #14437) 2017-10-07 21:59:04 +07:00
8e751a185c release 2017.10.07 2017-10-07 05:02:53 +07:00
3fc8f5b7c2 [ChangeLog] Actualize 2017-10-07 05:01:38 +07:00
665f42d8c1 [reddit] Sort formats (closes #14430) 2017-10-07 01:40:00 +07:00
e952847541 [PULL_REQUEST_TEMPLATE.md] Add explicit entry on flake8 2017-10-07 00:58:19 +07:00
b1a7bf44b9 [lnkgo] Relax _VALID_URL 2017-10-06 23:59:09 +07:00
2e2a8e97d5 [pornflip] Extend _VALID_URL (closes #14405) 2017-10-06 23:56:31 +07:00
ac93c09ab2 [xtube] Add support for embedded URLs (closes #14417) 2017-10-06 23:53:32 +07:00
cd6fc19ed7 [YoutubeDL] Ignore duplicates in --playlist-items
E.g. '--playlist-items 2-4,3-4,3' should result in '[2,3,4]', not '[2,3,4,3,4,3]'
2017-10-06 23:50:34 +07:00
86a15ed64b [test_YoutubeDL] Add test for #14425 2017-10-06 23:41:28 +07:00
7e85e8729f [YoutubeDL] Fix out of range --playlist-items for iterable playlists and reduce code duplication (closes #14425) 2017-10-06 23:34:46 +07:00
6be08ce602 [utils] Use in OnDemandPagedList by default
Not using cache results in redundant network I/O due to downloading the same pages while using --playlist-items n-m
2017-10-06 23:13:53 +07:00
cf5f6ed5be [xvideos] Add support for embed URLs and improve extraction (closes #14409) 2017-10-05 00:27:24 +07:00
6b46285e85 [comedycentral] new shortcut :theopposition for "The Opposition" show 2017-10-04 07:45:13 +02:00
6e736d86e7 [beeg] Fix extraction (closes #14403) 2017-10-04 04:27:42 +07:00
M.K
c110944fa2 [extractor/common] Fix typo in _parse_mpd_formats 2017-10-04 03:50:27 +07:00
9524dca3ac [README.md] Use revision bound link to YoutubeDL options (closes #14401) 2017-10-04 02:53:20 +07:00
3e4cedf9e8 [tvn24] Relax _VALID_URL 2017-10-03 23:28:13 +07:00
bfd484ccff Merge pull request #14392 from snipem/nbc-fix
Fix for JSON meta data download(closes #13651)
2017-10-03 14:49:55 +00:00
b7e14f06a4 Fix for JSON meta data download
Added fixes according to #13651 and user @remitamine
2017-10-03 15:17:28 +02:00
d2ae7e24e5 [postprocessor/ffmpeg] Convert to opus using libopus (closes #14381) 2017-10-02 04:43:25 +07:00
544ffb7790 [ketnet] Add support for videos without direct sources (closes #14377) 2017-10-02 04:15:12 +07:00
117589dfa2 [canvas] Generalize mediazone.vrt.be extractor and rework canvas and een 2017-10-02 04:14:36 +07:00
839728f5bf [afreecatv] Add support for adult videos (closes #14376) 2017-10-02 03:28:25 +07:00
fcdd37d053 release 2017.10.01 2017-10-01 21:54:11 +07:00
1dd126180e [ChangeLog] Actualize 2017-10-01 21:45:56 +07:00
4e599194d6 [tvp] Add support for new URL schema (closes #14368) 2017-10-01 18:59:00 +07:00
c5b7014a9c [generic] Add support for single format Video.js embeds (closes #14371) 2017-10-01 07:01:42 +07:00
c8da40d834 [yahoo] Bypass geo restriction for brightcove (#14210) 2017-10-01 04:49:27 +07:00
b69ca0ccfc [yahoo] Use extracted brightcove account id (closes #14210) 2017-10-01 04:37:42 +07:00
2c53bd51c6 [rtve:alacarta] Fix extraction (closes #14290) 2017-10-01 03:21:17 +07:00
3836b02ce8 [YoutubeDL] PEP 8 2017-09-30 22:56:40 +07:00
fa3fdeb41f [yahoo] Fix some tests 2017-09-30 22:54:22 +07:00
eb9a15be60 [yahoo] Add support for custom brigthcove embeds (closes #14210) 2017-09-30 22:47:03 +07:00
3600fd591d [YoutubeDL] Document youtube_include_dash_manifest 2017-09-28 00:46:48 +07:00
63d990d285 [generic] Add support for Video.js embeds 2017-09-28 00:37:30 +07:00
b14b2283a0 [gfycat] Add support for /gifs/detail URLs (closes #14322) 2017-09-27 22:48:47 +07:00
02d01e15f1 [generic] Fix infinite recursion for twitter:player URLs (closes #14339) 2017-09-26 21:47:18 +07:00
db96252831 [xhamsterembed] Fix extraction (closes #14308) 2017-09-24 19:23:08 +07:00
8b389f7e3c Credit the author of multiple generic HTML5 embeds fix 2017-09-24 18:21:38 +07:00
9fc41bcb6b release 2017.09.24 2017-09-24 00:22:50 +07:00
10cab6613f [ChangeLog] Actualize 2017-09-24 00:21:34 +07:00
4d182955a2 [kakao] Fix _VALID_URL 2017-09-24 00:19:27 +07:00
011da618bd [openload] Fix _load_cookies for python 2.6 2017-09-24 00:12:40 +07:00
4c54b89e03 Hide experimental phantomjs wrapper 2017-09-24 00:08:27 +07:00
a87d7b4953 Credit @nbppp2 for americastestkitchen (#13996) 2017-09-23 23:27:28 +07:00
2f3933aa1e Credit @ishitatsuyuki for mixcloud fix (#14132) 2017-09-23 23:26:35 +07:00
aab20aabfc Credit @jdong92 for voot (#14059) 2017-09-23 23:23:27 +07:00
16f54d0751 Credit @codeasashu for voot (#11814) 2017-09-23 23:20:20 +07:00
07d1344c85 Credit @coreynicholson for vlive:playlist (#13613) 2017-09-23 23:16:27 +07:00
47b5dfb047 Credit @luboss for joj (#13268) 2017-09-23 23:14:41 +07:00
e3440d824a [24video] Fix timestamp extraction and make non fatal (#14295) 2017-09-23 07:46:53 +07:00
136507b39a [24video] Add support for 24video.adult (closes #14295) 2017-09-23 07:41:22 +07:00
7f4921b38d [heise] PEP 8 2017-09-23 07:28:29 +07:00
f70ddd4aeb [kakao] Improve (closes #14007) 2017-09-23 07:28:24 +07:00
1c22d7a7f3 [kakao] Add extractor (closes #12298) 2017-09-23 07:28:19 +07:00
5c1452e8f1 [twitter] Add support for user_id-less URLs (closes #14270) 2017-09-23 06:38:09 +07:00
4bb58fa118 [americastestkitchen] Improve (closes #13996) 2017-09-23 06:29:20 +07:00
13de91c9e9 [americastestkitchen] Add extractor (closes #10764) 2017-09-23 06:29:07 +07:00
9ce1ac4046 [generic] Fix support for multiple HTML5 videos on one page (closes #14080) 2017-09-23 05:49:48 +07:00
095774e591 [mixcloud] Improve and simplify (closes #14132) 2017-09-23 05:37:03 +07:00
2384f5a64e [mixcloud] Fix extraction (closes #14088) 2017-09-23 05:36:57 +07:00
8c2895305d [options] Accept lrc as a subtitle conversion target format (closes #14292) 2017-09-23 02:30:03 +08:00
8c6919e433 [lynda] Add support for educourse.ga (closes #14286) 2017-09-21 23:00:35 +07:00
f6ff52b473 [beeg] Fix extraction (closes #14275) 2017-09-21 04:05:33 +07:00
12ea5c79fb [nbcsports:vplayer] Correct theplatform URL (closes #13873) 2017-09-21 02:53:06 +07:00
3b65a6fbf3 [twitter] Fix duration extraction 2017-09-20 03:58:06 +07:00
dc76eef092 [tvplay] Bypass geo restriction 2017-09-20 00:00:04 +07:00
8a1a60d173 [devscripts/check-porn] Fix gettestcases import 2017-09-19 22:51:20 +07:00
4d8c4b46d5 [heise] Add support for YouTube embeds 2017-09-17 22:46:52 +07:00
9c2a17f2ce [popcorntv] Add extractor (closes #5914, closes #14211) 2017-09-17 22:19:57 +07:00
4ed2d7b7d1 Fix flake8 issues after #14225 2017-09-17 13:53:04 +08:00
8251af63a1 [viki] Update app data (closes #14181) 2017-09-16 22:45:23 +07:00
790d379e4d [morningstar] Relax _VALID_URL 2017-09-16 22:39:46 +07:00
3869028ffb [utils] Use bytes-like objects in dfxp2srt
This fixes handling of non-UTF8 TTML subtitles

Closes #14191
2017-09-16 12:18:38 +08:00
68d43a61b5 Ignore TTML subtitles 2017-09-16 12:14:48 +08:00
a88d461dff Merge pull request #14225 from Tithen-Firion/openload-phantomjs-method
Openload phantomjs method
2017-09-16 02:28:28 +08:00
a4245acef8 [noovo] Fix extraction (closes #14214) 2017-09-15 23:12:19 +07:00
6be44a50ed [dailymotion:playlist] Relax _VALID_URL (closes #14219) 2017-09-15 22:25:38 +07:00
b763e1d68c [twitch] Add support for go.twitch.tv URLs (closes #14215) 2017-09-15 22:18:38 +07:00
cbf85239bb [vgtv] Relax _VALID_URL (closes #14223) 2017-09-15 22:13:30 +07:00
159d304a9f release 2017.09.15 2017-09-15 21:48:06 +07:00
86e55e317c [ChangeLog] Actualize 2017-09-15 21:45:18 +07:00
c46680fb2a [condenast] Fix extraction (closes #14196, closes #14207) 2017-09-15 02:01:17 +07:00
fad9fc537d [tv4] fix a test URL 2017-09-14 20:47:23 +02:00
0732a90579 [orf] Add new extractor for f4m stories 2017-09-14 20:37:46 +02:00
319fc70676 [tv4] Relax _VALID_URL (closes #14206) 2017-09-14 23:50:19 +07:00
e7c3e33456 [downloader/fragment] Restart inconsistent incomplete fragment downloads (#13731) 2017-09-14 23:19:53 +07:00
757984af90 Merge pull request #12909 from remitamine/raw-sub
[YoutubeDL] write raw subtitle files
2017-09-13 17:36:40 +08:00
2f483758bc [animeondemand] Improve and modernize 2017-09-11 04:32:35 +07:00
018cc61549 [animeondemand] Bypass geo restriction 2017-09-11 04:23:42 +07:00
2709d9fa28 [animeondemand] Add support for flash videos (closes #9944) 2017-09-11 04:23:42 +07:00
7dacceae75 release 2017.09.11 2017-09-11 03:30:33 +07:00
43df248f10 [ChangeLog] Actualize 2017-09-11 03:27:43 +07:00
f12a6e88b2 [rutube:playlist] Fix suitable (closes #14166) 2017-09-11 03:23:00 +07:00
806498cf2f release 2017.09.10 2017-09-10 22:16:55 +07:00
b98339b54b [ChangeLog] Actualize 2017-09-10 22:15:55 +07:00
bf6ec2fea9 [fox] Fix extraction (#14147) 2017-09-10 22:08:32 +07:00
c3dd44e085 [rutube] Use bool_or_none 2017-09-10 19:09:27 +07:00
c7e327c4d4 [utils] Introduce bool_or_none 2017-09-10 19:08:39 +07:00
48b813748d [rutube] Rework and generalize playlist extractors (closes #13565) 2017-09-10 18:40:33 +07:00
debed8d759 [rutube:playlist] Add extractor (closes #13534) 2017-09-10 18:40:33 +07:00
51aee72d16 [README.md] Clarify how to run extractor specific test cases 2017-09-08 22:13:17 +07:00
931edb2ada [radiocanada] Add fallback for title extraction 2017-09-08 21:53:24 +07:00
5113b69124 [abcnews,chilloutsoze,cracked,vice,vk] Use dedicated YouTube embeds extraction routines 2017-09-06 00:50:25 +07:00
66c9fa36c1 [youtube] Separate methods for embeds extraction 2017-09-06 00:48:37 +07:00
c5c9bf0c12 [YoutubeDL] Ensure dir existence for each requested format (closes #14116) 2017-09-05 23:31:34 +07:00
880fa66f4f [redtube] Fix formats extraction (closes #14122) 2017-09-05 22:45:49 +07:00
6348671c4a [arte] Relax unavailability check (closes #14112) 2017-09-04 23:08:40 +07:00
efc57145c1 [manyvids] Improve (closes #14059) 2017-09-03 17:32:23 +07:00
e9b865267a [manyvids] Add support for preview videos (closes #14053) 2017-09-03 17:31:53 +07:00
bc35f07537 [vidme:user] Make tests only matching (closes #14054) 2017-09-03 17:03:51 +07:00
0b4a8eb3ac [vidme:user] Relax _VALID_URLs 2017-09-03 17:03:45 +07:00
c1c1585b31 [bpb] Improve (closes #14086) 2017-09-03 16:43:33 +07:00
0cbb841ba9 [bpb] Fix extraction (closes #14043) 2017-09-03 16:39:12 +07:00
d7c7100e3d [soundcloud] Simplify and add test (closes #14093) 2017-09-03 16:29:58 +07:00
73602bcd0c [soundcloud] Fix download URL with private tracks 2017-09-03 16:28:34 +07:00
23b2df82c7 [aliexpress:live] Fix issues (closes #13698, closes #13707) 2017-09-03 16:05:31 +07:00
503115540d [aliexpress:live] Add extractor 2017-09-03 16:05:00 +07:00
64f0e30b93 [viidea] Capture and output lecture error message (#14099) 2017-09-02 15:44:49 +07:00
a3431e1224 [radiocanada] Skip unsupported platforms (closes #14100) 2017-09-02 15:33:54 +07:00
a2022b0c40 release 2017.09.02 2017-09-02 01:08:32 +07:00
8681ed7fc8 [ChangeLog] Actualize 2017-09-02 01:04:22 +07:00
8d81f3e36d [youtube] Force old layout for each webpage (closes #14083) 2017-09-02 00:58:19 +07:00
7998520933 [youtube] Fix upload date extraction (closes #14065) 2017-08-31 00:47:58 +07:00
5b4bfbfc3b [charlierose] Add support for episodes (closes #14062) 2017-08-30 23:50:33 +07:00
53647dfd0a [bbccouk] Add support for w-prefixed ids (closes #14056) 2017-08-30 05:27:56 +07:00
22f65a9efc Merge pull request #14048 from ryandesign/patch-1
Fix build failures with old cp and zip
2017-08-28 11:22:27 +08:00
c75c384fb6 Fix build failures with old cp and zip 2017-08-27 18:07:09 -05:00
1b41da488d [googledrive] Extend _VALID_URL (closes #9785) 2017-08-28 00:50:41 +07:00
fea82c1780 [googledrive] Add support for source format (closes #14046) 2017-08-28 00:39:22 +07:00
3902cdd0e3 [pornhd] Fix extraction (closes #14005) 2017-08-27 22:37:26 +07:00
2cfa7cbdd0 release 2017.08.27.1 2017-08-27 06:09:29 +07:00
cc0412ef91 [ChangeLog] Actualize 2017-08-27 06:06:49 +07:00
1c9c8de29e [youtube] Fix extraction with --youtube-skip-dash-manifest enabled (closes #14037) 2017-08-27 06:06:39 +07:00
f031b76065 release 2017.08.27 2017-08-27 04:28:04 +07:00
62c06c593d [ChangeLog] Actualize 2017-08-27 04:27:19 +07:00
ff17be3ac9 [extractor/generic] Extract from LD-JSON last of all
Previous sources may contain several formats, e.g. http://tamasha.com/v/PgGZ
2017-08-27 03:31:40 +07:00
1ed4549942 [extractor/common] Extract format id from label attribute of source tag for HTML5 videos (#14034) 2017-08-27 03:27:05 +07:00
dd121cc1ca [extractor/common] Extract height from res attribute of source tag for HTML5 videos (closes #14034) 2017-08-27 03:12:56 +07:00
a3c3a1e128 [http] Rework HTTP downloader
* Simplify code and split into separate routines to facilitate maintaining
* Make retry mechanism work on errors during actual download not only during connection establishment phase
* Retry on ECONNRESET and ETIMEDOUT during reading data from network
* Retry on content too short and various timeout errors
* Show error description on retry
* Closes #506, closes #809, closes #2849, closes #4240, closes #6023, closes #8625, closes #9483
2017-08-27 02:22:30 +07:00
085d9dd9be [rai] Fix audio formats extraction (closes #14024) 2017-08-26 22:02:49 +07:00
151978f38a [mixcloud] Fix extraction (closes #14020) 2017-08-26 19:32:57 +07:00
c7121fa7b8 [youtube] Fix controversy videos extraction (closes #14027, closes #14029) 2017-08-26 15:38:38 +07:00
745968bc72 [mixcloud] Fix extraction (closes #14015) 2017-08-24 22:28:44 +07:00
df235dbba8 release 2017.08.23 2017-08-23 23:23:13 +07:00
c4bdc68113 [ChangeLog] Actualize 2017-08-23 23:21:19 +07:00
5bae33485c [toutv] PEP 8 2017-08-23 22:50:00 +07:00
0830f3e048 [cbc:watch] Bypass geo-restriction (closes #13993) 2017-08-23 22:45:45 +07:00
8d7a24aff6 [toutv] Relax DRM check (closes #13994) 2017-08-23 22:28:09 +07:00
37d9af306a [googledrive] Simplify and carry long lines (#13638) 2017-08-23 00:33:53 +07:00
e01c3d2ef7 [extractor/common] Introduce _parse_xml 2017-08-23 00:32:41 +07:00
05915e379a [googledrive] Add support for subtitles (fixes #13619) 2017-08-22 23:48:59 +07:00
7b67b60773 Merge pull request #13669 from bmwiedemann/master
[build] Override timestamps in zip file
2017-08-22 21:51:20 +08:00
8d9c2a681a [pornhub] Relax uploader regex (closes #13906, closes #13975) 2017-08-21 23:06:27 +07:00
903d4d1625 [README.md] Switch to HTTPS URLs 2017-08-20 23:35:39 +07:00
8239c6791a [bandcamp:album] Extract track titles 2017-08-20 23:32:33 +07:00
b359e977b9 [extractor/common] Make HLS and DASH extraction non fatal in _parse_html5_media_entries (closes #13970) 2017-08-20 14:16:58 +07:00
305d99f0bd [build] Override timestamps in zip file
to make build reproducible.
See https://reproducible-builds.org/ for why this is good

Copying files to not interfere with freshness detection.
2017-08-19 21:43:48 +02:00
d3d45e0a45 [bbccouk] Add support for events URLs (closes #13893) 2017-08-19 23:54:15 +07:00
381ad4f309 [liveleak] Support multi-video pages (closes #6542) 2017-08-19 22:48:00 +08:00
e2481b9b6e [ChangeLog] Fix 2017-08-19 22:28:58 +08:00
09747ba766 [liveleak] Support another liveleak embedding pattern (closes #13336) 2017-08-19 22:28:13 +08:00
f8f18f332f [cda] Fix extraction (closes #13935) 2017-08-19 21:44:47 +08:00
95f3f7c20a [utils] Fix unescapeHTML for misformed string like "&a&quot;" (#13935) 2017-08-19 21:40:53 +08:00
f5469da9e6 [laola1tv] Add support for tv.ittf.com (closes #13965) 2017-08-19 19:48:20 +07:00
d14d9d8903 [mixcloud] Fix extraction (closes #13958) 2017-08-18 23:31:42 +07:00
ea004d34f8 release 2017.08.18 2017-08-18 01:05:27 +07:00
2738965d98 [ChangeLog] Actualize 2017-08-18 01:03:20 +07:00
4a91910365 [qqmusic:toplist] PEP 8 2017-08-18 01:00:07 +07:00
c0892b2b46 [arte] Detect unavailable videos (closes #13945) 2017-08-18 00:58:23 +07:00
a5ac0c4755 [YoutubeDL] Sanitize byte string format URLs (#13951) 2017-08-17 23:59:12 +07:00
5551d7714d [generic] Convert redirect URLs to unicode strings (closes #13951) 2017-08-17 23:58:01 +07:00
5f5c7b92dd [udemy] Fix paid course detection (#13943) 2017-08-17 23:14:46 +07:00
93d0583e34 [pluralsight] Use RPC API for course extraction (closes #13937) 2017-08-17 22:45:40 +07:00
5d28169747 Credit Genki Sky for clippit (bfabd17b33) 2017-08-17 21:21:17 +08:00
7ddab7742c [ChangeLog] Add an entry for Genki Sky's patch 2017-08-17 16:56:37 +08:00
bfabd17b33 Add new extractor 2017-08-17 16:56:06 +08:00
12f5304556 [ChangeLog] Add entry for #13805 2017-08-17 16:40:56 +08:00
25a6e769a1 [qqmusic] Fix tests and cleanup 2017-08-17 16:39:57 +08:00
d22b67f356 Merge pull request #13805 from gam2046/master
Fix QQ Music url changed
2017-08-17 16:11:35 +08:00
a1aa659662 [periscope] Renew HLS extraction (closes #13917) 2017-08-16 23:03:42 +07:00
4850478543 [extractor/common] Add support for float durations in _parse_mpd_formats (closes #13919) 2017-08-15 23:58:00 +07:00
134d85a7bd [qqmusic] review 2017-08-15 13:14:35 +08:00
5c037c0d1f [qqmusic]support QQMusicSingerIE 2017-08-15 13:14:35 +08:00
5d1bd3b907 [qqmusic]update valid url 2017-08-15 13:14:34 +08:00
19ada898dc fix QQ Music Url changed 2017-08-15 13:14:34 +08:00
da20951a57 [mixcloud] Extract decrypt key 2017-08-14 22:39:05 +07:00
16393d6535 release 2017.08.13 2017-08-13 08:58:30 +07:00
4f049e4aa8 [ChangeLog] Actualize 2017-08-13 08:00:15 +07:00
475bcb225f [pornhub:playlistbase] Skip videos from drop-down menu for all playlists (closes #12819, closes #13902) 2017-08-13 07:53:02 +07:00
b3c6515365 [fourtube] Add support for other sites (closes #6022, closes #7859, closes #13901) 2017-08-13 07:23:29 +07:00
eb02940cc7 [generic] Add test for #13895 2017-08-13 01:11:27 +07:00
4ef9152428 [limelight] Improve embeds detection (closes #13895) 2017-08-13 00:58:39 +07:00
0c43a481b9 [reddit] Add extractors (closes #13847) 2017-08-12 23:43:51 +07:00
868f79db41 [extractor/common] Fix _media_formats 2017-08-12 19:24:26 +07:00
70851a95c3 [aparat] Extract all formats (closes #13887) 2017-08-12 17:18:23 +07:00
e74e3b63e3 [YoutubeDL] Make sure format id is not empty 2017-08-12 17:14:11 +07:00
ac8491fcca [extractor/common] Make _family_friendly_search optional 2017-08-12 17:11:35 +07:00
82889d4ae5 [extractor/common] Respect source's type attribute for HTML5 media (closes #13892) 2017-08-12 16:48:11 +07:00
92a5c41532 [mixcloud] Fix play info decryption (closes #13885) 2017-08-12 16:30:50 +07:00
1663bd6e1c [generic] Replace vzaar embed test 2017-08-11 22:02:00 +07:00
41918eaa5c [generic] Add support for vzaar embeds 2017-08-11 22:00:39 +07:00
6ed99754bb release 2017.08.09 2017-08-09 23:52:22 +07:00
0e7dfa7d16 [ChangeLog] Actualize 2017-08-09 23:49:53 +07:00
baba5f4d1d [xxxymovies] Fix title extraction (closes #13868) 2017-08-09 23:46:49 +07:00
dee04d24a4 [nick] Add support for nick.com.pl (closes #13860) 2017-08-09 23:12:02 +07:00
5b3ddadcc3 [mixcloud] Fix play info decryption (closes #13867) 2017-08-09 22:55:13 +07:00
5b232f46dc [utils] Skip missing params in cli_bool_option (closes #13865) 2017-08-09 22:28:19 +07:00
4bf22f7a10 [20min] Fix embeds extraction 2017-08-08 05:41:38 +07:00
15d1e8a23d [dplayit] Fix extraction (closes #13851) 2017-08-07 22:43:42 +07:00
ee6a611665 [niconico] Support videos with multiple formats (closes #13522) 2017-08-07 00:19:46 +08:00
463e7216c8 [niconico] Support HTML5-only videos (closes #13806) 2017-08-06 23:07:28 +08:00
903a183b6a release 2017.08.06 2017-08-06 09:05:36 +07:00
92740e4241 [ChangeLog] Actualize 2017-08-06 09:02:14 +07:00
fac188c695 [pluralsight] Fix format selection 2017-08-06 08:44:28 +07:00
16afce174e [mpora] Remove extractor (closes #13826) 2017-08-06 08:18:16 +07:00
e2b4808fd8 [voot] Improve extraction (#10255, closes #11814) 2017-08-06 08:05:29 +07:00
daaaf5f594 [voot] Add extractor 2017-08-06 08:05:24 +07:00
f172c86dcd [vlive:channel] Limit number of videos per page to 100 (closes #13830) 2017-08-05 21:17:55 +07:00
1d5472290f [podomatic] Extend _VALID_URL (closes #13827) 2017-08-05 08:28:12 +07:00
c983cc3b71 [cinchcast] Extend _VALID_URL 2017-08-05 08:17:01 +07:00
1141e9104b Use relative paths for DASH fragments (closes #12990)
10x reduced JSON size
refs #13810
2017-08-05 07:40:29 +07:00
8519b88f67 [yandexdisk] Relax _VALID_URL (closes #13824) 2017-08-05 00:59:07 +07:00
bbbe1cebfc [mlb] Update test (closes #13777) 2017-08-05 00:09:36 +07:00
f31fd0693b [vidme] Extract DASH and HLS formats 2017-08-05 00:00:21 +07:00
799802f368 [teamfour] Remove extractor (closes #13782)
Now covered with generic extractor
2017-08-04 23:54:28 +07:00
b3b5870cba [pornhd] Fix extraction (closes #13783) 2017-08-04 23:51:03 +07:00
57a38a38c3 [udemy] Fix subtitles extraction (closes #13812) 2017-08-04 23:45:13 +07:00
11a6793f80 [mlb] Extend _VALID_URL (closes #13740) 2017-08-04 22:46:54 +07:00
1f03fef994 [README.md] Improve grammar 2017-08-04 22:43:44 +07:00
183062a4ab [pbs] Add support for new URL schema (closes #13801) 2017-08-03 23:19:59 +07:00
feee8d32e4 [phantomjs] add exe version to debug info 2017-08-03 14:17:25 +02:00
8cda78ef72 [test_YoutubeDL] Add a test for #10083 2017-08-02 23:12:34 +07:00
9118c9f18a [nrktv] Update API host (closes #13796) 2017-08-01 05:21:00 +07:00
5c9ea67bc0 release 2017.07.30.1 2017-07-30 20:47:31 +07:00
f701827e31 [ChangeLog] Actualize 2017-07-30 19:43:09 +07:00
8b9f50d7cb [watchbox] Add extractor (#13739) 2017-07-30 19:09:44 +07:00
0ed4758023 [clipfish] Remove extractor 2017-07-30 19:08:44 +07:00
a0a477b885 [youjizz] Fix extraction (closes #13744) 2017-07-30 15:48:22 +07:00
198d4cb40c [generic] Add support for another ooyala embed pattern (closes #13727) 2017-07-30 01:30:04 +07:00
ca127ab2c1 [ard] Add support for lives (closes #13771) 2017-07-29 23:07:28 +07:00
e445850e69 [soundcloud] Update client id 2017-07-29 18:45:57 +07:00
836ef26486 [soundcloud:trackstation] Add extractor (closes #13733) 2017-07-29 18:41:42 +07:00
c04017519d [svtplay] Use geo verification proxy for API request 2017-07-29 15:30:53 +07:00
2a7a823211 [svtplay] Update API URL (closes #13767) 2017-07-29 15:25:32 +07:00
95908ce453 [extractor/generic] PEP 8 2017-07-29 15:13:12 +07:00
cbbe66635f [yandexdisk] Add extractor (closes #13755) 2017-07-29 15:10:19 +07:00
c5a49ff084 [downloader/hls] Use redirect URL as manifest base (#13755) 2017-07-29 15:02:41 +07:00
24e966e8da [megaphone] Add extractor 2017-07-28 12:13:19 +02:00
9682666bda [amcnetworks] Make rating optional (closes #12453) 2017-07-27 02:04:51 +07:00
f9c48d895b [cloudy] Fix extraction (closes #13737) 2017-07-26 23:12:43 +07:00
c99d6890cb [nickru] Add extractor 2017-07-23 21:02:06 +07:00
70bfab0e9a [mtv] Improve thumbnal extraction 2017-07-23 21:02:06 +07:00
f0e31e32c9 [nick] Automate geo-restriction bypass (#13711) 2017-07-23 20:40:04 +07:00
3150976669 [ISSUE_TEMPLATE_tmpl.md] Minor improvements 2017-07-23 20:33:18 +07:00
e3ce912c3d [niconico] improve error reporting (#13696) 2017-07-23 16:25:30 +08:00
73095e013f [options] Typo 2017-07-23 16:24:18 +08:00
905d18a7aa [options] Correctly hide login info from debug outputs (#13696)
Iterate over opts instead of PRIVATE_OPTS for both performance and
correctness
2017-07-23 16:22:14 +08:00
0db492c02a release 2017.07.23 2017-07-23 01:09:09 +07:00
425f41319a [ChangeLog] Actualize 2017-07-23 01:06:08 +07:00
71dde5eecf [itv] Fix production id extraction (closes #13671) 2017-07-23 00:59:07 +07:00
935d6c20c0 [vidio] Make duration non fatal and fix typo 2017-07-23 00:44:50 +07:00
e0f1fb0a27 [mtv] Skip missing video parts (closes #13690) 2017-07-23 00:25:23 +07:00
0017d9ad6d [YoutubeDL] Improve default format specification (closes #13704) 2017-07-23 00:12:01 +07:00
327c8364f1 [sportbox:embed] Fix extraction 2017-07-22 21:35:14 +07:00
359aa2fdd1 [npo] Add support for npo3.nl URLs 2017-07-22 19:15:55 +07:00
f76c02c87b [dramafever] Fix tests 2017-07-22 11:41:40 +07:00
7d9a1db111 [dramafever] Remove video id from title (closes #13699) 2017-07-22 11:40:46 +07:00
0396806f67 [YoutubeDL] Do not override id, extractor and extractor_key in url_transparent
All these meta fields must be borrowed from final extractor that actually performs extraction.
This commit fixes extractor id in download archives for url_transparent downloads. Previously, 'transparent' extractor was erroneously
used for extractor archive id, e.g. 'eggheadlesson 4n8ugwwj5t' instead of 'wistia 4n8ugwwj5t'.
2017-07-21 00:13:32 +07:00
dc6520aa3d [egghead:lesson] Add extractor (#6635) 2017-07-20 23:22:36 +07:00
c653326a14 [funnyordie] Extract more metadata (closes #13677) 2017-07-20 22:50:56 +07:00
3fcf346ac1 [youku:show] Refine playlist extraction
Handle playlists that the initial page is not the first page
2017-07-20 23:20:46 +08:00
fa63cf6c23 [youku:show] Fix playlist extraction (closes #13248) 2017-07-20 22:57:51 +08:00
85f5a74b6c [tbs] Mark as broken and skip invalid tests 2017-07-20 21:19:09 +08:00
d20b1c6725 [dispeak] Recognize sevt subdomain (closes #13276) 2017-07-20 18:14:14 +08:00
bb176df3bb [spiegel:article] Move test 2017-07-17 22:19:40 +07:00
83d00044c1 [adn] Improve error reporting (#13663) 2017-07-16 20:50:32 +07:00
7abed4e06c [crunchyroll] Relax series and season regex (closes #13659) 2017-07-16 12:40:45 +07:00
13eb526f11 [nexx:embed] PEP 8 2017-07-16 05:23:19 +07:00
00d06e3cfc [spiegel:article] Add support for nexx iframe embeds (closes #13029) 2017-07-16 04:38:20 +07:00
749ca5eced [extractor/common] Fix playlist_from_matches 2017-07-16 04:33:14 +07:00
3f59b0154a [nexx:embed] Add extractor for iframe embeds 2017-07-16 04:32:37 +07:00
089b97cfee [nexx] Improve JS embed extraction 2017-07-16 04:30:48 +07:00
decf86044d [pearvideo] Improve (closes #13031) 2017-07-16 03:06:04 +07:00
94b817edeb [pearvideo] Add extractor 2017-07-16 03:02:31 +07:00
cea931a9e5 release 2017.07.15 2017-07-15 07:36:05 +07:00
ef78563e9c [ChangeLog] Actualize 2017-07-15 07:33:26 +07:00
961ea474b6 [YoutubeDL] PEP 8 2017-07-15 07:02:57 +07:00
ea3f20494f [youtube] PEP 8 2017-07-15 07:02:57 +07:00
c7604d79e9 [spiegeltv] Delegate extraction to nexx (closes #13159) 2017-07-15 07:02:57 +07:00
4e826cd9ae [nexx] Add extractor (closes #10807, closes #13465) 2017-07-15 07:02:57 +07:00
2583c0b54e Fix bugs caused by typos 2017-07-14 23:08:32 +07:00
7d02dcfaa2 [youtube] Don't capture YouTube Red ad for creator meta field (closes #13621) 2017-07-14 22:37:04 +07:00
00dbdfc1f7 [slideshare] Fix extraction 2017-07-14 22:11:07 +07:00
f354d84807 [5tv] Add another video URL pattern (closes #13354) 2017-07-14 22:10:17 +07:00
15da37c7dc [YoutubeDL] Don't expand env variables in meta fields (closes #13637) 2017-07-14 00:42:12 +07:00
9a0942ad55 [drtv] Make HLS and HDS extraction non fatal 2017-07-11 22:59:56 +07:00
f2bb33a986 [ted] Fix subtitles extraction (closes #13628, closes #13629) 2017-07-11 21:36:45 +07:00
3615bfe1b4 [twitter] Fix remaining tests 2017-07-11 16:46:37 +08:00
e8f20ffa03 [vine] Make sure the title won't be empty
And fix a relevant TwitterCard test case
2017-07-11 16:05:15 +08:00
9be31e771c [twitter] Support HLS streams in vmap URLs 2017-07-11 15:48:48 +08:00
7f176ac477 [periscope] Support pscp.tv URLs in embedded frames
And fix a relevant twitter test
2017-07-11 15:35:19 +08:00
2edfd745df [twitter] Extract mp4 urls via mobile API (closes #12726) 2017-07-11 15:19:36 +08:00
708f6f511e [niconico] Fix authentication error handling (closes #12486) 2017-07-11 15:04:45 +08:00
bb13949197 [niconico] Check login errors (#12486) 2017-07-11 15:03:11 +08:00
c3c94ca4a4 [giantbomb] Extract m3u8 formats (closes #13626) 2017-07-10 21:34:27 +08:00
e3cd1fcdd1 [vlive:playlist] Relax and simplify 2017-07-10 04:32:24 +07:00
b71c18b434 [vlive:playlist] Add extractor 2017-07-10 04:24:04 +07:00
7bf539edcc [eagleplatform] Fix test 2017-07-10 00:14:41 +07:00
65c416dda8 release 2017.07.09 2017-07-09 20:16:38 +07:00
207acd8465 [ChangeLog] Actualize 2017-07-09 20:15:15 +07:00
71a1db8919 [dailymail] Add support for embeds 2017-07-09 20:06:24 +07:00
6e925598d6 [csjw] Add coding cookie 2017-07-09 19:18:12 +07:00
73cf76a93f [joj] Rewrite and add support for generic embeds (closes #13268) 2017-07-09 19:17:54 +07:00
256a746d21 [joj] Add extractor 2017-07-09 19:17:38 +07:00
58179eb7d9 [abc.net.au:iview] Extract more formats (closes #13492, closes #13489) 2017-07-09 17:55:40 +07:00
485cb37576 [egghead:course] Improve (closes #13370) 2017-07-09 17:30:49 +07:00
ed84454d35 [egghead:course] Fix extraction 2017-07-09 17:30:25 +07:00
a02682fd13 Keep in sync with ffmpeg's current malformed AAC bitstream wording (closes #13587) 2017-07-09 17:09:44 +07:00
0d2f0b0357 [csjw] Make description optional 2017-07-09 17:05:11 +07:00
c319d1c483 [csjw] Fix issues and improve extraction (closes #13525) 2017-07-09 17:01:05 +07:00
d2b9f362fa [cjsw] Add extractor 2017-07-09 17:01:00 +07:00
4328ddf82b [extractor/common] Add support for AMP tags in _parse_html5_media_entries 2017-07-09 16:29:52 +07:00
250b042c7e [generic] Add tests for #13557 2017-07-09 16:02:38 +07:00
665e945246 [eagleplatform] Add support for referrer protected videos (closes #13557) 2017-07-09 15:57:58 +07:00
5af2fd7fa0 [eagleplatform] Add support for another embed pattern (#13557) 2017-07-09 15:55:04 +07:00
15237fcd51 [veoh] Extend _VALID_URL 2017-07-09 14:54:52 +07:00
7a57730907 [npo:live] Fix live stream id extraction (closes #13568) 2017-07-09 14:21:40 +07:00
8b347a389e [googledrive] Fix height extraction (closes #13603) 2017-07-09 00:26:13 +07:00
a49804816c [dailymotion] Add support for new layout (close #13580) 2017-07-08 18:12:15 +07:00
eadd313321 [yam] Remove extractor
mymedia.yam.com is dead. An wikipedia user also pointed out that Yam's
blog service is no longer available. [1]

[1] https://zh.wikipedia.org/zh-tw/%E5%A4%A9%E7%A9%BA%E9%83%A8%E8%90%BD
2017-07-08 15:48:05 +08:00
d852c6bc59 [xhamster] Extract all formats and fix duration extraction (#13593) 2017-07-07 22:49:11 +07:00
00e5c36315 [xhamster] Add support for new URL schema (closes #13593) 2017-07-07 22:27:34 +07:00
8a04ade86b Credit @parmjitv for #13322, #13503, #13541, #13549 2017-07-06 23:15:23 +07:00
ab328411d5 Credit @orng for ruv (#13396) 2017-07-06 23:15:16 +07:00
ddeff4be3f Credit @gfabiano for #13382, #13385, #13415 2017-07-06 23:15:09 +07:00
60d4401c5e [espn] Extend _VALID_URL (fixes #13244) 2017-07-06 22:55:59 +07:00
dee2ff1d81 [test_utils] Fix tests under Windows 2017-07-06 00:25:37 +07:00
6554708252 [kaltura] Fix typo in subtitles extraction (closes #13569) 2017-07-05 23:20:50 +07:00
0a2e1b2e30 [vier] Adapt extraction to redesign (#13575) 2017-07-05 22:52:47 +07:00
babbc04d45 [xuite] Move to the new HTML5 API and reduce # of requests 2017-07-05 23:27:12 +08:00
609ff8ca19 [utils] Support attributes with no values in get_elements_by_attribute() 2017-07-05 23:27:12 +08:00
b6c9fe4162 release 2017.07.02 2017-07-02 20:17:10 +07:00
4d9ba27bba [ChangeLog] Actualize 2017-07-02 20:12:40 +07:00
50ae3f646e [thisoldhouse] Add more fallbacks for video id (closes #13541) 2017-07-02 20:06:15 +07:00
99a7e76240 [thisoldhouse] Update test 2017-07-02 20:05:11 +07:00
a3a6d01a96 [thisoldhouse] Fix video id extraction (closes #13540) 2017-07-02 20:04:51 +07:00
02d61a65e2 [xfileshare] Extend format regex (closes #13536) 2017-07-02 08:00:22 +07:00
9b35297be1 [extractors] Add import for tastytrade 2017-07-01 18:39:29 +07:00
4917478803 [ted] Fix extraction (closes #13535)) 2017-07-01 18:39:01 +07:00
54faac2235 [tastytrade] Add extractor (closes #13521) 2017-06-30 22:20:30 +07:00
c69701c6ab [extractor/common] Improve _json_ld 2017-06-30 22:19:06 +07:00
d4f8ce6e91 [dplayit] Relax video id regex (closes #13524) 2017-06-30 21:55:45 +07:00
b311b0ead2 [generic] Extract more generic metadata (closes #13527) 2017-06-30 21:42:04 +07:00
72d256c434 [bbccouk] Extend _VALID_URL 2017-06-29 22:29:28 +07:00
b2ed954fc6 [bbccouk] Capture and output error message (closes #13518) 2017-06-29 22:27:53 +07:00
a919ca0ad6 [cbsnews] Actualize test 2017-06-28 22:30:12 +07:00
88d6b7c2bd [cbsnews] Relax video info regex (fixes #13284) 2017-06-28 22:21:35 +07:00
fd1c5fba6b [facebook] Add test for plugin video embed (#13493) 2017-06-27 22:38:59 +07:00
0646e34c7d [facebook] Add support for plugin video embeds and multiple embeds (closes #13493) 2017-06-27 22:38:54 +07:00
bf2dc9cc6e [soundcloud] Fix tests 2017-06-27 21:26:46 +07:00
f1c051009b [soundcloud] Switch to https for API requests 2017-06-27 21:20:18 +07:00
33ffb645a6 [pandatv] Switch to https for API and download URLs 2017-06-26 22:11:09 +07:00
35544690e4 [pandatv] Add support for https URLs 2017-06-26 22:00:31 +07:00
136503e302 [ChangeLog] Update after #13494 2017-06-26 19:56:07 +08:00
4a87de72df [niconico] fix sp subdomain links 2017-06-25 21:30:05 +02:00
a7ce8f16c4 release 2017.06.25 2017-06-25 05:16:06 +07:00
a5aea53fc8 [ChangeLog] Actualize 2017-06-25 05:13:12 +07:00
0c7a631b61 [adobepass] Add support for ATTOTT MSO (DIRECTV NOW) (closes #13472) 2017-06-25 05:03:17 +07:00
fd9ee4de8c [wsj] Add support for barrons.com (closes #13470) 2017-06-25 02:15:35 +07:00
5744cf6c03 [ign] Add another video id pattern (closes #13328) 2017-06-25 01:59:15 +07:00
9c48b5a193 [raiplay:live] Improve and add test (closes #13414) 2017-06-25 01:49:27 +07:00
449c665776 [raiplay:live] Add extractor 2017-06-25 01:48:54 +07:00
23aec3d623 [redbulltv] Restore hls format prefix 2017-06-25 01:10:31 +07:00
27449ad894 [redbulltv] Add support for lives and segments (closes #13486)) 2017-06-25 01:09:12 +07:00
bd65f18153 [onetpl] Add support for videos embedded via pulsembed (closes #13482) 2017-06-24 18:33:31 +07:00
73af5cc817 [YoutubeDL] Skip malformed formats for better extraction robustness 2017-06-23 21:18:33 +07:00
b5f523ed62 [ooyala] Add test for missing stream['url']['data'] 2017-06-23 20:56:48 +07:00
4f4dd8d797 [ooyala] Make more robust 2017-06-23 20:56:21 +07:00
4cb18ab1b9 [ooyala] Skip empty format URLs (closes #13471, closes #13476) 2017-06-23 20:50:48 +07:00
ac7409eec5 [hgtv.com:show] Fix typo 2017-06-23 02:54:12 +07:00
170719414d release 2017.06.23 2017-06-23 02:13:21 +07:00
38dad4737f [ChangeLog] Actualize 2017-06-23 02:10:54 +07:00
ddbb4c5c3e [youtube] Adapt to new automatic captions rendition (closes #13467) 2017-06-23 02:00:19 +07:00
fa3ea7223a [hgtv.com:show] Relax video config regex and update test (closes #13279, closes #13461) 2017-06-23 00:42:42 +07:00
0f4a5a73e7 [drtuber] Fix formats extraction (fixes 12058) 2017-06-23 00:08:36 +07:00
18166bb8e8 [youporn] Fix upload date extraction 2017-06-22 00:47:02 +07:00
d4893e764b [youporn] Improve formats extraction 2017-06-22 00:40:15 +07:00
97b6e30113 [youporn] Fix title extraction (closes #13456) 2017-06-22 00:20:45 +07:00
9be9ec5980 [googledrive] Fix formats' sorting (closes #13443) 2017-06-20 22:58:33 +07:00
048b55804d [watchindianporn] Fix extraction (closes #13411) 2017-06-20 04:30:45 +07:00
6ce79d7ac0 [abcotvs] Fix test md5 2017-06-20 04:07:00 +07:00
1641ca402d [vimeo] Add fallback mp4 extension for original format 2017-06-20 01:27:59 +07:00
85cbcede5b [ruv] Improve, extract all formats and metadata (closes #13396) 2017-06-19 23:46:03 +07:00
Orn
a1de83e5f0 [ruv] Add extractor 2017-06-19 23:45:45 +07:00
fee00b3884 [viu] Fix extraction on older python 2.6 2017-06-19 22:57:37 +07:00
2d2132ac6e [adobepass] Fix extraction on older python 2.6 2017-06-19 22:54:53 +07:00
cc2ffe5afe [pandora.tv] Fix upload_date extraction (closes #12846) 2017-06-19 16:20:36 +08:00
560050669b [asiancrush] Add extractor (closes #13420) 2017-06-18 20:18:51 +07:00
eaa006d1bd release 2017.06.18 2017-06-18 00:16:49 +07:00
a6f29820c6 [ChangeLog] Actualize 2017-06-18 00:15:43 +07:00
1433734c35 [downloader/common] Use utils.shell_quote for debug command line 2017-06-17 23:50:21 +07:00
aefce8e6dc [utils] Use compat_shlex_quote in shell_quote 2017-06-17 23:48:58 +07:00
8b6ac49ecc [postprocessor/execafterdownload] Encode command line (closes #13407) 2017-06-17 23:16:53 +07:00
b08e235f09 [compat] Fix compat_shlex_quote on Windows (closes #5889, closes #10254) 2017-06-17 23:14:24 +07:00
be80986ed9 [postprocessor/metadatafromtitle] Fix missing optional meta fields (closes #13408) 2017-06-17 19:05:10 +07:00
473e87064b [devscripts/prepare_manpage] Fix deprecated escape sequence on py36 2017-06-17 17:37:25 +08:00
4f90d2aeac [Makefile] Excluding __pycache__ correctly (#13400) 2017-06-17 17:09:24 +08:00
b230fefc3c [polskieradio] Fix extraction 2017-06-16 04:57:56 +07:00
96a2daa1ee [extractor/common] Improve jwplayer subtitles extraction 2017-06-15 23:40:39 +07:00
0ea6efbb7a [xfileshare] Add support for fastvideo.me 2017-06-15 21:41:49 +07:00
6a9cb29509 [extractor/common] Fix json dumping with --geo-bypass
The line "[debug] Using fake IP %s (%s) as X-Forwarded-For." was printed
to stdout even with -j/-J, which breaks the resultant JSON.
2017-06-15 13:04:36 +08:00
ca27037171 [bilibili] Fix extraction of videos with double quotes in titles
Closes #13387
2017-06-15 11:19:03 +08:00
0bf4b71b75 [4tube] Fix extraction (closes #13381) 2017-06-15 04:16:50 +07:00
5215f45327 [disney] Add support for disneychannel.de 2017-06-15 04:13:04 +07:00
0a268c6e11 [extractor/common] Improve jwplayer formats extraction (closes #13379) 2017-06-14 22:02:15 +07:00
7dd5415cd0 [npo] Improve _VALID_URL (closes #13376) 2017-06-14 21:33:40 +07:00
b5dc33daa9 [corus] Add support for showcase.ca 2017-06-13 23:27:27 +07:00
97fa1f8dc4 [corus] Add support for history.ca (closes #13359) 2017-06-13 23:16:21 +07:00
b081f53b08 [compat] Add compat_HTMLParseError to __all__ 2017-06-12 02:36:43 +07:00
cb1e6d8985 release 2017.06.12 2017-06-12 02:23:17 +07:00
9932ac5c58 [ChangeLog] Actualize 2017-06-12 02:01:15 +07:00
bf87c36c93 [xfileshare] PEP 8 2017-06-12 02:01:12 +07:00
b4a3d461e4 [utils] Handle HTMLParseError in extract_attributes (closes #13349) 2017-06-12 01:52:24 +07:00
72b409559c [compat] Introduce compat_HTMLParseError 2017-06-12 01:50:32 +07:00
534863e057 [xfileshare] Add support for rapidvideo (closes #13348) 2017-06-12 00:16:47 +07:00
16bc958287 [xfileshare] Modernize and pass referrer 2017-06-12 00:14:04 +07:00
624bd0104c [rutv] Add support for testplayer.vgtrk.com (closes #13347) 2017-06-11 21:36:19 +07:00
28a4d6cce8 [newgrounds] Extract more metadata (closes #13232) 2017-06-11 21:30:06 +07:00
2ae2ffda5e [utils] Improve unified_timestamp 2017-06-11 21:27:22 +07:00
70e7967202 [newgrounds:playlist] Add extractor (closes #10611) 2017-06-11 20:50:55 +07:00
6e999fbc12 [newgrounds] Improve formats and uploader extraction (closes #13346) 2017-06-11 19:44:44 +07:00
7409af9eb3 [msn] Fix formats extraction 2017-06-11 08:56:53 +07:00
4e3637034c [extractor/generic] Ensure format id is unicode string 2017-06-10 23:56:20 +07:00
1afd0b0da7 [extractor/common] Return unicode string from _match_id 2017-06-09 00:40:03 +07:00
7515830422 [turbo] Ensure format id is string 2017-06-09 00:31:56 +07:00
f5521ea209 [sexu] Ensure height is int 2017-06-09 00:30:23 +07:00
34646967ba [jove] Ensure comment count is int 2017-06-09 00:29:20 +07:00
e4d2e76d8e [golem] Ensure format id is string 2017-06-09 00:27:11 +07:00
87f5646937 [gfycat] Ensure filesize is int 2017-06-09 00:24:23 +07:00
cc69a3de1b [foxgay] Ensure height is int 2017-06-09 00:22:14 +07:00
15aeeb1188 [flickr] Ensure format id is string 2017-06-09 00:20:07 +07:00
1693bebe4d [sohu] Fix numeric fields 2017-06-09 00:16:42 +07:00
4244a13a1d [safari] Improve authentication detection (closes #13319) 2017-06-08 23:20:48 +07:00
931adf8cc1 [liveleak] Ensure height is int (closes #13313) 2017-06-08 22:54:30 +07:00
c996943418 [YoutubeDL] Sanitize more fields (#13313) 2017-06-08 22:53:14 +07:00
76e6378358 [README.md] Improve man page formatting 2017-06-08 22:02:42 +07:00
a355b57f58 [README.md] Clarify output template references (closes #13316) 2017-06-08 21:52:19 +07:00
1508da30c2 [streamango] Skip download for test (closes #13292) 2017-06-07 21:53:40 +07:00
eb703e5380 [streamango] Make title optional 2017-06-07 21:53:33 +07:00
0a3924e746 [rtlnl] Improve _VALID_URL (closes #13295) 2017-06-06 21:21:44 +07:00
e1db730d86 [tvplayer] Fix extraction (closes #13291) 2017-06-06 00:13:57 +07:00
537191826f release 2017.06.05 2017-06-05 00:48:07 +07:00
130880ba48 [ChangeLog] Actualize 2017-06-05 00:43:38 +07:00
f8ba3fda4d Credit @jktjkt for dvtv formats (#13063) 2017-06-05 00:38:44 +07:00
e1b90cc3db Credit @mikf for beam:vod (#13032) 2017-06-05 00:35:41 +07:00
43e6579558 Credit @adamvoss for bandcamp:weekly (#12758) 2017-06-04 23:22:19 +07:00
6d923aab35 [bandcamp:weekly] Improve and extract more metadata (closes #12758) 2017-06-04 23:21:30 +07:00
62bafabc09 [bandcamp:weekly] Add extractor 2017-06-04 23:21:07 +07:00
9edcdac90c [pornhub:uservideos] Add missing raise 2017-06-04 20:39:55 +07:00
cd138d8bd4 [pornhub:playlist] Fix extraction (closes #13281) 2017-06-04 15:54:19 +07:00
cd750b731c [godtv] Remove extractor (closes #13175) 2017-06-03 22:08:12 +07:00
4bede0d8f5 [YoutubeDL] Don't emit ANSI escape codes on Windows 2017-06-03 19:14:23 +07:00
f129c3f349 [safari] Fix typo (closes #13252) 2017-06-02 01:03:51 +07:00
39d4c1be4d [youtube] Improve chapters extraction (closes #13247) 2017-06-01 23:29:45 +07:00
f7a747ce59 [1tv] Lower preference for http formats (closes #13246) 2017-06-01 22:19:52 +07:00
4489d41816 [francetv] Relax _VALID_URL 2017-06-01 00:15:15 +07:00
87b5184a0d [drbonanza] Fix extraction (closes #13231) 2017-05-31 23:56:32 +07:00
c56ad5c975 [packtpub] Fix authentication(closes #13240) 2017-05-31 15:44:29 +01:00
6b7ce85cdc [README.md] Mention http_dash_segments protocol 2017-05-30 23:50:48 +07:00
d10d0e3cf8 [README.md] Add an example for how to use .netrc on Windows
That's a Python bug: http://bugs.python.org/issue28334
Most likely it will be fixed in Python 3.7: https://github.com/python/cpython/pull/123
2017-05-29 14:58:07 +08:00
941ea38ef5 release 2017.05.29 2017-05-29 00:42:18 +07:00
99bea8d298 [ChangeLog] Actualize 2017-05-29 00:33:56 +07:00
a49eccdfa7 [youtube] Parse player_url if format URLs are encrypted or DASH MPDs are requested
Fixes #13211
2017-05-28 20:20:20 +08:00
a846173d93 [xhamster] Simplify (closes #13216) 2017-05-28 07:55:56 +07:00
78e210dea5 [xhamster] Fix author and like/dislike count extraction 2017-05-28 07:55:07 +07:00
8555204274 [xhamster] Extract categories (closes #11728) 2017-05-28 07:50:15 +07:00
164fcbfeb7 [abcnews] Improve and remove duplicate test (closes #12851) 2017-05-28 07:06:56 +07:00
bc22df29c4 [abcnews] Add support for embed URLs 2017-05-28 07:06:29 +07:00
7e688d2f6a [gaskrank] Improve (closes #12493) 2017-05-28 06:47:38 +07:00
5a6d1da442 [gaskrank] Fix extraction 2017-05-28 06:47:30 +07:00
703751add4 [medialaan] PEP 8 (closes #12774) 2017-05-28 06:27:57 +07:00
4050be78e5 [medialaan] Fix videos with missing videoUrl
A rough trick to get around the two different json styles medialaan seems to be using.
Fix for these example videos:
https://vtmkzoom.be/video?aid=45724
https://vtmkzoom.be/video?aid=45425
2017-05-28 06:27:52 +07:00
4d9fc40100 [dvtv] Improve and fix playlists support (closes #13063) 2017-05-28 06:19:54 +07:00
765522345f [dvtv] Parse adaptive formats as well
The old code hit an error when it attempted to parse the string
"adaptive" for video height. Actually parsing the returned playlists is
a good idea because it adds more output formats, including some
audio-only-ones.
2017-05-28 06:19:46 +07:00
6bceb36b99 [beam] Improve and add support for mixer.com (closes #13032) 2017-05-28 05:43:04 +07:00
1e0d65f0bd [beam:vod] Add extractor 2017-05-28 05:42:23 +07:00
03327bc9a6 [cbsinteractive] Relax _VALID_URL (closes #13213) 2017-05-27 22:37:24 +07:00
b407d8533d [utils] Drop an compatibility wrapper for Python < 2.6
addinfourl.getcode is added since Python 2.6a1. As youtube-dl now
requires 2.6+, this is no longer necessary.

See 9b0d46db11
2017-05-27 23:05:02 +08:00
20e2c9de04 [adn] fix formats extraction 2017-05-26 20:00:44 +01:00
d16c0121b9 [youku] Extract more metadata (closes #10433) 2017-05-27 00:08:37 +08:00
7f4c3a7439 [cbsnews] Fix extraction (closes #13205) 2017-05-26 22:42:27 +07:00
28dbde9cc3 release 2017.05.26 2017-05-26 22:29:16 +07:00
cc304ce588 [ChangeLog] Actualize 2017-05-26 22:27:56 +07:00
98a0618941 [ChangeLog] Update after the fix for #11381 2017-05-26 23:22:54 +08:00
fd545fc6d1 Revert "[youtube] Don't use the DASH manifest from 'get_video_info' if 'use_cipher_signature' is True (#5118)"
This reverts commit 87dc451108.
2017-05-26 23:22:54 +08:00
97067db2ae [bbc] Add support for authentication 2017-05-26 22:12:24 +07:00
c130f0a37b [tudou] Merge into youku extractor (fixes #12214)
Also, there are no tudou playlists anymore. All playlist URLs points to youku
playlists.
2017-05-26 23:04:42 +08:00
d3d4ba7f24 [youku:show] Fix extraction 2017-05-26 21:59:16 +08:00
5552c9eb0f [utils] Recognize more patterns in strip_jsonp()
Used in Youku Show pages
2017-05-26 21:58:18 +08:00
59ed87cbd9 [youku] Fix extraction (closes #13191) 2017-05-26 19:16:52 +08:00
b7f8749304 [udemy] Fix extraction for outputs' format entries without URL (closes #13192) 2017-05-25 22:28:26 +07:00
5192ee17e7 [postprocessor/ffmpeg] Fix metadata filename handling on Python 2
Fixes #13182
2017-05-25 22:07:03 +08:00
e834f04400 [vimeo] Fix formats' sorting (closes #13189) 2017-05-24 22:58:16 +07:00
884d09f330 [cbsnews] fix extraction for 60 Minutes videos 2017-05-24 09:44:41 +01:00
9e35298f97 Merge pull request #12861 from Tithen-Firion/cbsinteractive-fix
[cbsinteractive] update extractor and test cases
2017-05-24 10:21:33 +02:00
0551f1b07b Credit @gritstub for vevo fix (#12879) 2017-05-23 21:47:40 +07:00
de53511201 Credit @timendum for rai (#11790) and mediaset (#12964) 2017-05-23 00:41:53 +07:00
2570e85167 release 2017.05.23 2017-05-23 00:17:48 +07:00
9dc5ab041f [ChangeLog] Actualize 2017-05-23 00:15:44 +07:00
01f3c8e290 Credit @fredbourni for noovo (#12792) 2017-05-23 00:08:14 +07:00
06c1b3ce07 Credit @mphe for streamango (#12643) 2017-05-23 00:07:31 +07:00
0b75e42dfb Credit @zurfyx for atresplayer improvements (#12548) 2017-05-23 00:00:49 +07:00
a609e61a90 [downloader/external] Pass -loglevel to ffmpeg downloader (closes #13183) 2017-05-22 23:40:07 +07:00
afdb387cd8 [streamcz] Add support for subtitles 2017-05-21 15:41:52 +07:00
dc4e4f90a2 [youtube] Modernize 2017-05-21 01:18:56 +07:00
fdc20f87a6 [youtube] Fix DASH manifest signature decryption (closes #8944) 2017-05-21 01:11:37 +07:00
35a2d221a3 [toggle] Relax _VALID_URL (closes #13172) 2017-05-20 23:06:30 +07:00
daa4e9ff90 [adobepass] Add support for Brighthouse MSO 2017-05-20 20:50:46 +07:00
2ca29f1aaf [toypics] Improve and modernize 2017-05-20 01:29:33 +07:00
77d682da9d [toypics] Fix extraction 2017-05-20 01:18:03 +07:00
8fffac6927 [njpwworld] Fix extraction (closes #13162) 2017-05-19 23:11:02 +07:00
5f6fbcea08 [hitbox] Add support for smashcast.tv (closes #13154) 2017-05-19 22:34:00 +07:00
00cb0faca8 [mitele] Update app key regex 2017-05-19 21:54:57 +07:00
bfdf6fcc66 release 2017.05.18.1 2017-05-18 23:00:03 +07:00
bcaa1dd060 [ChangeLog] Actualize 2017-05-18 22:58:14 +07:00
0e2d626ddd [jsinterp] Fix typo and cleanup regexes (closes #13134) 2017-05-18 22:57:38 +07:00
9221d5d7a8 [ChangeLog] Fix typo 2017-05-18 22:37:32 +07:00
9d63e57d1f release 2017.05.18 2017-05-18 22:30:37 +07:00
3bc1eea0d8 [ChangeLog] Actualize 2017-05-18 22:29:25 +07:00
7769f83701 [jsinterp] Add support for quoted names and indexers (closes #13123, closes #13130) 2017-05-18 22:18:33 +07:00
650bd94716 [vier] Relax regexes and extract more metadata (closes #12539) 2017-05-17 23:39:01 +07:00
36b226d48f [vier] Extract more info
Extract the `episode_number` and `upload_date`. Also extract the real
`description`.
2017-05-17 23:38:50 +07:00
f2e2f0c777 [extractor/common] Fix rtmp and rtsp formats' URLs in _extract_wowza_formats 2017-05-17 22:20:25 +07:00
6f76679804 [extractor/common] Add support for schemeless URLs in _extract_wowza_formats (closes #13088, closes #13092) 2017-05-16 22:11:34 +07:00
7073015a23 [vier] PEP 8 and cleanup 2017-05-15 22:00:53 +07:00
89fd03079b [vier] Improve extraction
+ Add support for authentication
* Bypass authentication when no credentials provded
* Improve extraction robustness
2017-05-15 21:46:55 +07:00
1c45b7a8a9 [dailymail] Fix sources extraction (closes #13057) 2017-05-14 12:47:19 +07:00
60f5c9fb19 [utils] Recognize more audio codecs (#13081) 2017-05-14 12:33:33 +07:00
c360e641e9 [dailymotion] Extend _VALID_URL (closes #13079) 2017-05-14 09:55:40 +07:00
6f3c632c24 release 2017.05.14 2017-05-14 07:38:40 +07:00
09b866e171 [ChangeLog] Actualize 2017-05-14 07:37:31 +07:00
166d12b00c [options] PEP 8 2017-05-14 07:33:24 +07:00
2b8e6a68f8 [extractor/generic] Add test for mediaset embed 2017-05-14 06:40:19 +07:00
d105a7edc6 [mediaset] Fix upload date 2017-05-14 06:39:47 +07:00
5d29af3d15 [extractor/generic] Add support for mediaset embeds 2017-05-14 06:29:16 +07:00
ca04de463d [mediaset] Add support for shortcut 2017-05-14 06:28:40 +07:00
946826eec7 [extractor/generic] Remove duplicate limelight code 2017-05-14 06:17:34 +07:00
76d5a36391 [extractor/common] Respect Width and Height attributes in ISM manifests 2017-05-14 06:11:45 +07:00
56f9c77f0e [mediaset] Improve extraction (closes #12708, closes #12964) 2017-05-14 05:30:13 +07:00
0de136341a [mediaset] Add extractor 2017-05-14 05:30:02 +07:00
1339ecb2f8 [orf:radio] Cleanup _VALID_URLs (closes #11643) 2017-05-14 04:31:20 +07:00
efe9316703 [orf:radio] Fix extraction
Since oe1.orf.at has been updated, both ORF radios supported by youtube_dl
use the same API. This commit honors this fact by merging both extractors
into one.
2017-05-14 04:31:14 +07:00
851a01aed6 [aljazeera] Extend _VALID_URL 2017-05-14 00:57:02 +07:00
b845766597 [imdb] Relax _VALID_URL (closes #13056) 2017-05-14 00:32:50 +07:00
fa26734e07 [postprocessor/metadatafromtitle] Add support regex syntax for --metadata-from-title (closes #13065) 2017-05-14 00:03:15 +07:00
12f01118b0 [francetv] Add support for mobile.france.tv (closes #13068) 2017-05-13 21:57:00 +07:00
7fc60f4ee9 [upskill] Add extractor (closes #13043) 2017-05-13 21:52:59 +07:00
58bb440283 [extractor/generic] Extract wistia embed code into separate method 2017-05-13 21:51:58 +07:00
7ad4362357 [thescene] fix extraction(closes #13061) 2017-05-12 16:37:09 +01:00
6c52477f59 [condenast] improve embed support 2017-05-12 16:37:09 +01:00
116283ff64 [liveleak] Fix extraction (#12053) 2017-05-12 19:15:33 +08:00
7274f3d0e9 [douyu] Support Douyu shows (closes #12228) 2017-05-12 18:44:24 +08:00
3166b1f0ac [myspace] Improve _VALID_URL (closes #13040) 2017-05-10 22:35:46 +07:00
39ee263819 use platform=desktop in assets url(closes #13041) 2017-05-10 08:50:30 +01:00
a7ed6b341c release 2017.05.09 2017-05-09 04:20:13 +07:00
cbd84b5817 [ChangeLog] Actualize 2017-05-09 23:17:22 +07:00
6d1ded7502 [francetv] Adapt to site redesign (closes #13034) 2017-05-09 23:07:01 +07:00
5d0968f0af [packtpub] add support for authentication(closes #12622) 2017-05-09 11:15:14 +01:00
8d65880e24 [drtv] Improve extraction and update tests (closes #13013, closes #13016) 2017-05-09 15:37:09 +07:00
b972fb037b [drtv] Lower preference for SignLanguage formats (closes #13013) 2017-05-09 15:36:02 +07:00
5996d21aea [cspan] add support for brightcove live embeds(closes #13028) 2017-05-09 00:51:12 +01:00
afa0200bf0 [vrv] extract dash formats and subtitles 2017-05-08 20:04:40 +01:00
e9137224b3 [YoutubeDL] Force restrict filenames when no locale is set for python 2 as well (#13027) 2017-05-09 01:14:02 +07:00
804181dda9 [funimation] remove codes related to old login method and update test 2017-05-08 18:58:13 +01:00
8fa17117df [funimation] fix authentication(closes #13021) 2017-05-08 18:13:58 +01:00
3b859145c2 [adultswim] Fix Extraction(closes #8640)(closes #10950)(closes closes #11042)(closes #12121)
- add support for adobe pass authentication
- add support for live streams
- add support for show pages
2017-05-08 15:07:40 +01:00
04c09f1961 [turner] extract thumbnail and is_live and strip description 2017-05-08 15:07:40 +01:00
bf82b87323 [nonktube] Use econfig nuevo URL 2017-05-08 20:13:22 +07:00
b6eb74e340 [nonktube] Add extractor (closes #8647, closes #13024) 2017-05-08 20:10:39 +07:00
3d40084b83 [nuevo] Pass headers to _extract_nuevo 2017-05-08 20:03:38 +07:00
52294cdda7 [nbc] remove unused imports and extract permalink from modified urls 2017-05-07 09:31:14 +01:00
2eeb588efe [nbc] improve extraction(closes #12364) 2017-05-07 08:59:31 +01:00
4ac0f573ef release 2017.05.07 2017-05-07 04:51:34 +07:00
3892a9f4ab [ChangeLog] Actualize 2017-05-07 04:44:54 +07:00
3995d37da5 [youtube] Fix TFA (#12927) 2017-05-07 04:19:11 +07:00
e4a75d7932 [test_youtube_chapters] PEP 8 2017-05-07 00:00:11 +07:00
e00eb564e9 [youtube] Fix authentication (closes #12927) 2017-05-06 23:58:47 +07:00
10c87c151b [utils] Rename try_multipart_encode to _multipart_encode_impl
To state that this is an internal function and people should be careful
when using it outside youtube-dl.
2017-05-06 19:06:18 +08:00
228cd9bb90 [bilibili] Fix video downloading (closes #13001) 2017-05-06 18:58:38 +08:00
566fbbaefd [rmcdecouverte] Improve (closes #12937) 2017-05-06 17:56:10 +07:00
74c09c852a [rmcdecouverte] Fix extraction 2017-05-06 17:56:10 +07:00
fd178b8748 [theplatform] extract chapters 2017-05-06 07:19:07 +01:00
a57a8e9918 [test_youtube_chapters] Add coding cookie 2017-05-06 05:30:56 +07:00
1f9fefe7f5 [crackle] Update test 2017-05-06 03:39:14 +07:00
8b4774dcac [bandcamp] Fix thumbnail extraction 2017-05-06 03:35:42 +07:00
a99cc4ca16 [pornhub] Extend _VALID_URL (closes #12996) 2017-05-06 02:46:37 +07:00
9cafc3fd8b [youtube] Extract chapters 2017-05-06 02:27:06 +07:00
329e3dd5ad [nrk] Extract chapters 2017-05-05 22:59:15 +07:00
1d9e0a4f40 [vice] update tests and add support for ooyala embeds in article pages 2017-05-05 16:13:12 +01:00
7ad53cb7ff [laola1tv] PEP 8 2017-05-05 21:59:23 +07:00
b2ad479d17 [utils] Fix multipart_encode for Python < 3.5 2017-05-05 20:51:59 +08:00
4ac6dc3732 [vice] Support Vice articles (closes #12968) 2017-05-05 20:26:51 +08:00
cc7bda4fff [vice] Fix extraction for non en_us videos (closes #12967) 2017-05-05 20:01:02 +08:00
50ad078b7b [gdcvault] Fix extraction for videos with gdc-player.html
Closes #12733
2017-05-05 15:13:40 +08:00
4947f13cd0 [pbs] Improve multipart video support (closes #12981) 2017-05-04 22:42:49 +07:00
7f09e523e8 [laola1tv:embed] Fix tests 2017-05-04 22:41:47 +07:00
4fe14732a2 [laola1tv] fix extraction(closes #12880) 2017-05-04 16:07:08 +01:00
ff6f9a6704 [extractor/common] fix typo in _extract_akamai_formats 2017-05-04 16:07:08 +01:00
c89267d31a Merge branch 'master' into openload-phantomjs-method 2017-05-04 11:00:06 +02:00
0c26548601 [cda] Implement birthday verification (closes #12789) 2017-05-04 16:26:17 +08:00
5401bea27f [leeco] Fix extraction (closes #12974)
Seems on mobile devices a similar API is used, but I always get an AD
with mimicking that API.
2017-05-04 03:18:56 +08:00
7a6d33a9a5 [pbs] extract chapters information 2017-05-02 20:41:48 +01:00
fa2a36d9bc [ffmpeg] add support for chapters field postprocessing 2017-05-02 20:41:48 +01:00
55949fede6 [common] introduce chapters field 2017-05-02 20:41:48 +01:00
7fc875195f [amp] imporove thumbnail and subtitle extraction 2017-05-02 00:06:58 +01:00
c6fe5a7e12 [douyutv] Update test 2017-05-02 03:45:27 +08:00
ae21d2fd94 [dotsub] Update test 2017-05-02 02:56:44 +08:00
77481f1386 [democracynow] Update test 2017-05-02 01:38:31 +07:00
d86d169dd5 [dailymotion] Add working test 2017-05-02 01:37:23 +07:00
b9f9f361fa [crunchyroll] Update test 2017-05-02 00:56:51 +07:00
ab39a25c75 [foxsports] fix extraction(closes #12945) 2017-05-01 09:02:41 +01:00
a146fa1c68 [coub] Update test and remove comment count extraction 2017-05-01 05:54:44 +07:00
e0c1e9a98c release 2017.05.01 2017-05-01 01:39:52 +07:00
086041e2f8 [ChangeLog] Actualize 2017-05-01 01:34:51 +07:00
74da856544 [infoq] Make audio format extraction non fatal (closes #12938) 2017-05-01 01:23:05 +07:00
9edf47df7b [brightcove] Allow whitespace around attribute names in embedded code 2017-05-01 01:03:47 +07:00
238cec17ae [extractor/anvato] PEP 8 2017-04-30 22:04:21 +07:00
50534b7158 [downloader/fragment] PEP 8 2017-04-30 22:04:01 +07:00
9cd4209724 [zaq1] Improve extraction (closes #12693) 2017-04-30 21:46:05 +07:00
33a81c2c6f [extractor/common] Extract view count from JSON-LD 2017-04-30 21:45:59 +07:00
deef31955b [utils] Improve unified_timestamp
Seen at http://zaq1.pl/video/xev0e
2017-04-30 21:45:53 +07:00
9dac2cec2d [zaq1] Add new extractor 2017-04-30 21:45:47 +07:00
6ec371cd9e [xvideos] Extract og:duration (closes #12828) 2017-04-30 18:14:01 +07:00
13081db1f5 [xvideos] Add video duration 2017-04-30 18:10:49 +07:00
b07ea5eaec [vevo] Modernize 2017-04-30 17:58:22 +07:00
5599253009 [vevo] Fix extraction (config.token.key) 2017-04-30 17:56:10 +07:00
98ce1a3fd3 [utils] add video/mp2t to mimetype2ext 2017-04-30 09:03:10 +01:00
5ff1bc0cc1 [YoutubeDL] write raw subtitle files 2017-04-29 20:03:03 +01:00
ba5c3caf88 [washingtonpost] Fix invalid escape sequence on Python 3.6 2017-04-30 02:15:28 +08:00
b5c39537be [noovo] Improve extraction (closes #12792) 2017-04-30 00:24:25 +07:00
1c7c76e4fb [noovo] Add extractor 2017-04-30 00:24:19 +07:00
557194591a [washingtonpost] Add support for embeds (closes #12699) 2017-04-29 23:07:26 +07:00
27e70a8f6c Merge pull request #12869 from Tithen-Firion/cbc-update-tests
[cbc] update test cases
2017-04-29 21:34:18 +08:00
a4c81e4968 [yandexmusic:playlist] Fix extraction for python 3 (closes #12888) 2017-04-29 20:23:26 +07:00
7986c3abcd [anvato] Improve extraction (closes #12913)
* Promote to regular shortcut based extractor
* Add mcp to access key mapping table
* Add support for embeds extraction
* Add support for anvato embeds in generic extractor
2017-04-29 19:49:04 +07:00
a1ebfd4494 Merge pull request #12854 from Tithen-Firion/appletrailer-test-fix
[appletrailers] update test cases
2017-04-29 19:24:38 +08:00
7552f96352 [openload] Add required version 2017-04-29 12:41:57 +02:00
98f9d87381 [phantomjs] Add required version checking 2017-04-29 12:41:42 +02:00
fcace2d1ad [openload] raise not found before executing js 2017-04-29 10:30:45 +02:00
d19093bd50 Merge pull request #12906 from Tithen-Firion/clean-html-fix
[utils] Fix inconsistent output of clean_html
2017-04-29 15:58:45 +08:00
24eb7c2578 [xtube] Fix extraction with non-standard JSON 'sources'
Closes #12734

Thanks @paulguy for the fix!
2017-04-29 15:55:08 +08:00
e7db6759e4 [downloader/external] Properly handle live stream downloading cancellation (closes #8932) 2017-04-29 04:33:35 +07:00
b364c87c42 [tvplayer] Fix extraction (closes #12908) 2017-04-29 03:46:08 +07:00
9222d94510 [test_utils] Add one more clean_html test 2017-04-28 18:05:14 +02:00
edd9221cd2 [utils] Fix inconsistent output of clean_html
`\s` in Python 2.x doesn't match unicode whitespace characters by
default
2017-04-28 17:34:27 +02:00
bc8a2ea071 release 2017.04.28 2017-04-28 18:30:03 +07:00
7527923371 [ChangeLog] Actualize 2017-04-28 18:27:29 +07:00
20783b8b50 [aenetworks] fix extraction for shows with single season 2017-04-28 12:04:56 +01:00
bf2a5555c0 [go] add support for Disney, DisneyJunior and DisneyXD show pages 2017-04-28 09:48:52 +01:00
fb8e8b2d16 [adobepass] use geo verification headers for all requests 2017-04-28 09:48:52 +01:00
b62985a9a5 [youtube] Recognize another HTML5 player URL (#12885) 2017-04-28 16:25:04 +08:00
e31fed95b4 [youtube] Recognize new locale-based player URLs (fixes #12885) 2017-04-28 15:48:30 +08:00
3fd0f70f6a [cbslocal] Update test 2017-04-28 04:26:59 +07:00
33c62efc32 [collegerama] Update tests 2017-04-28 04:00:49 +07:00
6b4ddd336c [afreecatv] Fix title extraction 2017-04-28 04:00:15 +07:00
c12b4b80f8 [archiveorg] Update test 2017-04-28 03:48:32 +07:00
064fafe932 [appleconnect] Update test 2017-04-28 03:47:25 +07:00
ac1a5b9a12 [audioboom] Update test 2017-04-28 03:36:28 +07:00
a15777491a [atresplayer] Update test 2017-04-28 03:32:25 +07:00
d8571dd6bf [bleacherreport] Update tests 2017-04-28 03:28:26 +07:00
c0fa4245ce [downloader/fragment] Remove assert for resume_len when no fragments downloaded
This may be incorrect due some header (e.g. flv header in f4m downloader)
2017-04-28 03:26:19 +07:00
8814ae42bc [beeg] Update test 2017-04-28 03:14:11 +07:00
0f63dc2402 [bandcamp] Update test 2017-04-28 03:13:12 +07:00
dde97ea8da [canalc2] Update test 2017-04-28 03:07:42 +07:00
30bb6ce1a4 [test_InfoExtractor] Fix test_parse_m3u8_formats 2017-04-28 03:01:43 +07:00
c89b49f743 [extractor/common] Add manifest_url for explicit group rendition formats 2017-04-28 03:00:14 +07:00
6f4a888416 [br] Update test 2017-04-28 02:53:11 +07:00
f5edd7ae51 [clipfish] Update test 2017-04-28 02:51:30 +07:00
96820c1c6b [cbsinteractive] extract formats with CBSIE 2017-04-27 20:23:52 +02:00
c95e2b5911 [cbc] update test cases 2017-04-27 18:07:07 +02:00
374560f018 [test_download] Fix order when testing file's md5 2017-04-27 22:27:34 +07:00
ff99fe529e Don't list master m3u8 playlists in format list (closes #12832) 2017-04-27 21:53:17 +07:00
e095109da1 [cbsinteractive] update test cases 2017-04-27 15:40:17 +02:00
d68afc5bc9 [cbsinteractive] fix extractor 2017-04-27 15:27:01 +02:00
76c1951036 [appletrailers] update test cases 2017-04-27 10:04:21 +02:00
e8bfe2a946 [streamable] Add support for new embedded URL schema 2017-04-26 23:39:53 +07:00
3dc8b61b7f [arte:+7] Relax _VALID_URL (closes #12837) 2017-04-26 01:55:29 +07:00
a82f41841d release 2017.04.26 2017-04-26 00:06:12 +07:00
30a4ab191a [ChangeLog] Actualize 2017-04-26 00:03:13 +07:00
ac9c69ace7 [extractor/common] Improve jwplayer regex 2017-04-25 23:46:05 +07:00
85f6de25e4 [downloader/fragment] Clarify current_fragment's index and mark as experimental 2017-04-25 23:33:35 +07:00
538eee7b6a Add missing test m3u8 file 2017-04-25 22:26:30 +07:00
9f54ae2873 Ignore and clean *.ytdl files 2017-04-25 22:42:55 +08:00
01cb57016f [iqiyi] Fix extraction of Yule videos 2017-04-25 22:23:57 +08:00
40e41780f1 [phantomjs] add cookie support 2017-04-25 15:12:54 +02:00
da57ebaf84 [openload] separate PhantomJS code from extractor 2017-04-25 01:06:14 +02:00
290f64dbaa [downloader/fragment] Improve .ytdl format and start documenting 2017-04-24 23:50:20 +07:00
adb4b03cd5 [downloader/fragment] Don't process ytdl file when it's not needed yet 2017-04-24 23:05:56 +07:00
0eee52f34b Introduce --keep-fragments 2017-04-24 03:09:08 +07:00
d3f0687cf7 [downloader/fragment] Use temp file for current fragment 2017-04-24 02:54:17 +07:00
a4d6cf970c [YoutubeDL] Fix output template for missing timestamp (closes #12796) 2017-04-24 00:50:39 +07:00
3019cb0c99 [extractor/common] Rephrase comment 2017-04-23 11:52:07 +07:00
ddd258f922 [test_InfoExtractor] Add m3u8 parsing test for NAME attribute in EXT-X-STREAM-INF tag 2017-04-23 11:49:57 +07:00
07ad0cf34f [vidio] Improve and sort formats 2017-04-23 11:48:51 +07:00
9c99bef704 [extractor/common] Use float for scaled tbr 2017-04-23 11:33:49 +07:00
ffbc8386b9 [brightcove] match only video elements with data-video-id attribute 2017-04-22 22:26:20 +01:00
4abdba643c [downloader/fragment] remove unused code 2017-04-22 18:19:47 +01:00
3e0304fe6e [downloader/fragment] use the documented names for fragment progress_hooks fields 2017-04-22 16:42:24 +01:00
fbf56be213 [iqiyi] Fix playlist detection (#12504) 2017-04-22 22:11:37 +08:00
54f54fcca7 [socks] Report errors elegantly when credentails are required but missing
In some non-standard implementations, the server may respond AUTH_USER_PASS
even if's not listed in available authentication methods. (it should
respond AUTH_NO_ACCEPTABLE per standards)
2017-04-22 21:48:41 +08:00
facfd79f9a [azubu] Remove extractor as the site is gone (closes #12813) 2017-04-22 21:20:25 +08:00
3110bb937d [porn91] Fix extraction (closes #12814) 2017-04-22 21:16:36 +08:00
cb2520802d [extractor/common] Improve m3u8 extraction (closes #12211)
* Extract m3u8 parsing to separate method
* Improve rendition groups extraction
* Build stream name according stream GROUP-ID
* Ignore reference to AUDIO group without URI when stream has no CODECS
+ Add test coverage for parsing m3u8 from #11507, #11995, #12211 and twitch vod
2017-04-22 07:01:00 +07:00
f779958250 [vidzi] Fix extraction (closes #12793) 2017-04-21 23:37:06 +07:00
8abc7dca39 [amp] extract error message(closes #12795) 2017-04-20 05:16:41 +01:00
ea0c2f219c [downloader/fragment] use a general file to store fragment download context 2017-04-19 18:53:15 +01:00
481ef51e23 [brightcove] PEP 8 2017-04-19 21:47:03 +07:00
5b995f713b [utils] add support for ttml styles 2017-04-19 14:38:40 +01:00
75a2485407 [fragment,hls,f4m,dash,ism] improve fragment downloading
- resume immediately
- no need to concatenate segments and decrypt them on every resume
- no need to save temp files for segments

and for hls downloader:
- no need to download keys for segments that already downloaded
2017-04-19 11:46:07 +01:00
58f6ab72ed [odnoklassniki] update tests 2017-04-19 00:16:55 +01:00
2dc48df5bc [xfileshare] Add support for gorillavid.com and daclips.com (closes #12776) 2017-04-18 23:58:37 +07:00
18848d226a [instagram] Fix extraction (closes #12777) 2017-04-18 22:40:26 +07:00
a32a9a7ef5 [extractor/common] Add support multiple getters in try_get 2017-04-18 22:39:58 +07:00
bae1404893 [extractor/common] Add support for video of WebPage context in _json_ld (closes #12778) 2017-04-18 22:21:38 +07:00
06d0ad9a4e [brightcove] Support URLs with bcpid instead of playerID
Fixes #12482
2017-04-18 23:04:22 +08:00
f631b55791 [brightcove] Fix _extract_url (closes #12782) 2017-04-18 21:46:25 +07:00
bf1b87cd91 [common] Relax JWPlayer regex and remove duplicate urls(#12768) 2017-04-17 08:48:24 +01:00
1c35b3da44 [odnoklassniki] extract m3u8 formats 2017-04-16 21:27:08 +01:00
16a09aefe3 release 2017.04.17 2017-04-17 00:46:32 +07:00
fa7a6e6de6 [ChangeLog] Actualize 2017-04-17 00:43:56 +07:00
ab87c26009 [extractor/generic] Add test for #12761 2017-04-17 00:36:59 +07:00
91bc57e4c5 [limelight] Add support for channels and channels lists embedded using LimelightEmbeddedPlayerFlash (#12761) 2017-04-17 00:33:47 +07:00
e5d39886ec [limelight] Improve embeds extraction (closes #12761)
* Move extraction code to extractor
* Add extraction for LimelightEmbeddedPlayerFlash embeds
* Extract multiple video
2017-04-17 00:23:16 +07:00
751c89a27d [itv] Extract series metadata 2017-04-16 23:19:20 +07:00
c2d7d76efd [itv] Fix rtmp formats (#12759) 2017-04-16 23:15:24 +07:00
f67177cae8 [itv] Use native hls 2017-04-16 21:52:45 +07:00
5935ef3c5d [itv] Lower preference for rtmp formats (closes #12759) 2017-04-16 21:52:07 +07:00
47e0cef46e [openload] rewrite extractor 2017-04-16 00:34:34 +02:00
1183e22c7e Credit @triple-j for extracting more metadata from go90.com(#12721) 2017-04-15 23:19:23 +01:00
72950c4dce Merge pull request #12752 from triple-j/go90_improvements_pull_request
[go90] Improve extraction
2017-04-15 23:48:13 +02:00
334f41e0d8 [go90] Improve extraction
- add metadata for 'series', 'episode', 'season', 'season_id',
  'season_number', and 'episode_number'
- integrate series title into the title
- extract subtitles  (fallback to `vtt` if the subtitle file type
  detection fails as that is the most likely extension)
2017-04-15 16:32:00 -05:00
022250a594 release 2017.04.16 2017-04-16 01:49:02 +07:00
76b5f99617 [ChangeLog] Actualize 2017-04-16 01:47:58 +07:00
d35dc344af [YoutubeDL] Apply expand_path after output template substitution
os.path.expandvars translates '%%' into '%' making output template invalid.
Before: '%%(ext)s' -(expand path)-> '%(ext)s' -(outtmpl subst.)-> 'mp4'
After: '%%(ext)s' -(outtmpl subst.)-> '%(ext)s' -(expand path)-> '%(ext)s'
2017-04-16 01:39:07 +07:00
51350db5a3 [test_YoutubeDL] Add test for #11163 2017-04-16 01:14:05 +07:00
0563f7ac6e [YoutubeDL] Propagate overridden metadata to IE results of type url (closes #11163) 2017-04-16 00:57:36 +07:00
413c1f8e2f [extractor/generic] Extract RSS entries as url_transparent (#11163) 2017-04-16 00:57:32 +07:00
8068296276 [streamango] Improve extraction (closes #12643) 2017-04-15 21:50:51 +07:00
4db79fa1bc [streamango] Add extractor 2017-04-15 21:50:46 +07:00
b2a19e3829 [wsj] Improve and modernize (closes #12558) 2017-04-15 20:59:05 +07:00
3266d08af2 [wsj:article] Add extractor 2017-04-15 20:58:57 +07:00
0254f93b08 [brightcove] Validate ambiguous embeds' URLs (closes #9163, closes #12005, closes #12178, closes #12480) 2017-04-15 20:31:49 +07:00
40158f55c9 [brightcove] Relax video tag embeds extraction
BrightcoveNewIE.extract_urls(): Handle player data when it is all in
attributes of the <video> tag, and also the id is
data-brightcove-video-id not data-video-id.

Add test to generic extractor, note HEAD gives 404, no checksum computed(?).
2017-04-15 20:31:43 +07:00
3dfceb286c [udemy] Add support for react rendition (closes #12744) 2017-04-15 03:26:19 +07:00
57e2cb6c3a [curiositystream] PEP 8 2017-04-15 02:33:26 +07:00
6f1b2374af [canalplus] PEP 8 2017-04-15 02:33:12 +07:00
d24bd4ffdb release 2017.04.15 2017-04-15 01:39:33 +07:00
b9859e2735 [ChangeLog] Actualize 2017-04-15 01:37:26 +07:00
34c3bbe1bd [youku] Keep old fileid extraction code as fallback (#12741) 2017-04-15 01:28:02 +07:00
e5106ba402 [youku] Fix fileid extraction 2017-04-14 13:23:02 -05:00
38d2f8325f release 2017.04.14 2017-04-14 00:31:49 +07:00
6f4dd6667b [ChangeLog] Actualize 2017-04-14 00:29:36 +07:00
95728fda70 [aenetworks] Add support for specials (closes #12723) 2017-04-13 23:39:55 +07:00
3f7409f745 [hbo] extract m3u8 formats 2017-04-13 12:28:22 +01:00
b2fff30817 [go90] Add new extractor(closes #10127) 2017-04-13 12:28:22 +01:00
f59746372a [downloader/hls] add basic support for EXT-X-BYTERANGE tag(#10955) 2017-04-13 12:28:22 +01:00
34d98cc411 [tv2hu] Fix invalid escape sequence on Py3.6 2017-04-13 16:52:52 +08:00
40fcba5edb improve coding style 2017-04-12 20:38:43 +01:00
e4d74e2778 [tv2hu] improve extraction 2017-04-12 19:44:32 +01:00
3ef1d0c733 [tv2hu] Add new extractor 2017-04-12 19:36:02 +01:00
3962260b7d Credit @SkiTheSlicer for Verizon adobe pass support (#12721) 2017-04-12 16:54:53 +01:00
0ee79a378a [generic] exclude urls with xml ext from valid video urls(closes #10768)(closes #11654) 2017-04-12 16:22:08 +01:00
79a51069e5 [adobepass] improve comcast and verison login code(closes #10803) 2017-04-12 15:19:58 +01:00
a9a346535b [adobepass] Add Verizon support (#10803) 2017-04-12 01:18:57 -04:00
89beedd31f [youtube] improve m3u8 format extraction 2017-04-11 21:55:59 +01:00
e109f1ff43 [afreecatv] Fix extraction for videos with different key layout (closes #12718) 2017-04-12 02:17:34 +07:00
d23028a8fb [youtube] Remove explicit preference for audio-only and video-only formats
In order not to break sorting when new formats appear
2017-04-11 22:41:48 +07:00
6214611a4a [canalplus] Bypass geo restriction 2017-04-11 22:30:15 +07:00
1730878167 release 2017.04.11 2017-04-11 02:17:53 +07:00
689cd458a6 [ChangeLog] Actualize 2017-04-11 02:16:20 +07:00
6b9466de2f [afreecatv] Fix extraction (closes #12706) 2017-04-11 02:05:53 +07:00
61568e50cf [generic] Add support for <object> youtube embeds (closes #12637) 2017-04-11 00:04:32 +07:00
364a69e8c6 [test_download] Fix testing playlists with single video and add comments 2017-04-11 00:01:02 +07:00
6240925b40 [bbccouk] Treat bitrate as audio+video bitrate in media selector 2017-04-10 22:56:22 +07:00
964744af95 [bbccouk] Skip unrecognized formats in media selector (#12701) 2017-04-10 22:53:51 +07:00
1af959ef9f [bbccouk] Add support for https protocol in media selector (closes #12701) 2017-04-10 22:53:06 +07:00
a206ef62df [curiositystream] fix extraction(closes #12638) 2017-04-10 13:50:08 +01:00
3f2ce6896a [adn] update subtitle decryption key 2017-04-09 12:33:29 +01:00
a6f7263cf4 [chaturbate] Fix extraction (closes #12665) 2017-04-09 01:39:40 +07:00
4372436504 release 2017.04.09 2017-04-09 00:01:28 +07:00
eb8cc8ea3b [ChangeLog] Actualize 2017-04-08 23:59:17 +07:00
41b263ac8a [canvas] Add IE_DESC (closes #12478) 2017-04-08 22:45:45 +07:00
ca8fca9d9d [vrt] Add IE_DESC (closes #12477) 2017-04-08 22:44:31 +07:00
e129fa0846 [vier] Add IE_DESC 2017-04-08 22:43:29 +07:00
2bd875edfe [medici] Add extractor (closes #3406) 2017-04-08 22:38:37 +07:00
95152630db [rbmaradio] Add support for redbullradio.com URLs 2017-04-08 21:39:07 +07:00
04e431cf97 [npo:live] Improve (closes #12555) 2017-04-08 21:31:22 +07:00
1591ba258a [npo:live] Add support for default url 2017-04-08 21:30:38 +07:00
29c6726646 [mixcloud] Fix view count extraction and modernize 2017-04-08 21:11:08 +07:00
a66e25859a [mixcloud:playlist] Relax title extraction and fix description extraction (closes #12582) 2017-04-08 21:04:09 +07:00
c93c0fc2fd [mixcloud:playlist] Fix title extraction 2017-04-08 20:47:07 +07:00
90e3f18fc1 [thesun] Extract playlists (closes #11298, closes #12674) 2017-04-08 20:08:31 +07:00
5f3e0b69ef [TheSun] Add new extractor 2017-04-08 19:54:04 +07:00
28b674ca23 [ceskateleveize:porady] Add extractor (closes #7411, closes #12645) 2017-04-08 19:46:42 +07:00
e18f1da97a [ceskateleveize] Improve extraction and remove URL replacement hacks 2017-04-08 19:41:14 +07:00
78280352ca [kaltura] Cleanup regexes from redundant escaping 2017-04-08 16:48:27 +07:00
a01825a541 [kaltura] Add support for iframe embeds 2017-04-08 16:40:57 +07:00
f8f2da25ab [wshh] Extract html5 entries and delegate to generic extractor (closes #12676) 2017-04-08 16:01:56 +07:00
4c03973296 [airmozilla] Fix extraction (closes #12670) 2017-04-08 15:39:58 +08:00
60e5016199 [test_download] Remove unused import 2017-04-08 14:19:01 +07:00
c4d6fc6d65 [test_subtitles] Fix raiplay test 2017-04-08 14:11:20 +07:00
1b3feca0a7 [raiplay] Extract subtitles 2017-04-08 14:11:03 +07:00
80b2fdf9ac [test_download] Match info dicts against tests before matching info file 2017-04-08 14:10:12 +07:00
3bef10a50c [test_download] typo in comment 2017-04-08 13:07:25 +07:00
a84da06f49 [test_download] Improve diagnostic on wrong 'id' 2017-04-08 13:07:25 +07:00
3461f5db06 [xfileshare] Add support for vidlo.us (closes #12660) 2017-04-06 23:39:09 +07:00
0378b8b917 [xfileshare] Add support for vidbom.com (closes #12661) 2017-04-06 23:38:16 +07:00
7f04386b89 [aenetworks] Add more video URL regexes (closes #12657) 2017-04-06 02:36:48 +07:00
fac39cccd4 [odnoklassniki] fix format sorting for 1080p quality 2017-04-03 23:39:56 +01:00
b68e00b08a [rtl2] add support for you.rtl2.de(closes #10257) 2017-04-03 21:36:35 +01:00
2ab0bfcd81 [vshare] Add extractor (closes #12278) 2017-04-04 03:05:18 +07:00
b022f4f600 release 2017.04.03 2017-04-03 03:53:55 +07:00
e2435ba5f3 [ChangeLog] Actualize 2017-04-03 03:52:44 +07:00
a9bb61a425 [discoveryvr] Add new extractor(closes #12578) 2017-04-02 09:22:09 +01:00
dbf70c489f [tv5mondeplus] clean description and use stable id 2017-04-02 00:26:48 +01:00
61e2331ad8 [tv5mondeplus] Add new extractor(closes #11386) 2017-04-01 23:49:40 +01:00
fd47550885 [extractor/common] Add coding cookie 2017-04-02 04:42:10 +07:00
4457823dda [extractor/common] Move censorship checks to a separate method and add check for just another ISP 2017-04-02 03:57:44 +07:00
b3633fa0ce [pericope] Add support for pscp.tv URLs 2017-04-02 03:20:28 +07:00
b56e41a701 release 2017.04.02 2017-04-02 02:39:15 +07:00
a76c25146a [ChangeLog] Actualize 2017-04-02 02:37:18 +07:00
361f293ab8 [rai] Skip not found content item id 2017-04-02 02:24:13 +07:00
b8d8cced9b [rai] Improve extraction (closes #11790)
* Fix georestriction detection
* Detect live streams
+ Extract relinker metadata
* Improve ContentItem detection
+ Extract series metadata
* Fix tests
2017-04-02 02:14:42 +07:00
51342717cd [rai] Fix extraction 2017-04-02 02:10:53 +07:00
48ab554feb [vrv] add support for series pages 2017-04-01 18:09:36 +01:00
a6f3a162f3 [limelight] improve extraction for audio only formats 2017-04-01 15:35:39 +01:00
91399b2fcc [funimation] fix extraction(closes #10696)(#11773) 2017-04-01 13:33:04 +01:00
eecea00d36 [xfileshare] Add support for vidabc.com (closes #12589) 2017-04-01 18:56:35 +07:00
2cd668ee59 [xfileshare] Improve extraction and extract hls formats 2017-04-01 18:55:48 +07:00
ca77b92f94 [crunchyroll] pass geo verifcation proxy 2017-04-01 09:33:23 +01:00
e97fc8d6b8 [cwtv] extract ISM formats 2017-04-01 07:50:24 +01:00
be61efdf17 [tvplay] Bypass geo restriction 2017-04-01 07:26:40 +01:00
77c8ebe631 [vrv] Add new extractor 2017-03-31 23:29:23 +01:00
7453999580 [packtpub] Add extractor (closes #12610) 2017-04-01 00:25:27 +07:00
1640eb0961 [YoutubeDL] Return early when extraction of url_transparent fails 2017-03-31 23:57:35 +07:00
3e943cfe09 [generic] pass base_url to _parse_jwplayer_data 2017-03-31 14:54:06 +01:00
82be732b17 [adn] Add new extractor 2017-03-31 12:24:23 +01:00
639e5b2a84 [allocine] Extract more metadata 2017-03-29 04:43:12 +07:00
128244657b [allocine] Fix extraction 2017-03-29 05:23:20 +08:00
12ee65ea0d [options] Mention ISM for --fragment-retries and --skip-unavailable-fragments 2017-03-28 23:35:48 +07:00
aea1dccbd0 [openload] fix extractor 2017-03-29 00:00:09 +08:00
9e691da067 release 2017.03.26 2017-03-26 08:11:40 +07:00
82eefd0be0 [ChangeLog] Actualize 2017-03-26 23:39:12 +07:00
f7923a4c39 [ChangeLog] Update after #12307 2017-03-26 22:07:12 +08:00
cc63259d18 Merge pull request #12307 from rndusr/fix/str-item-assignment
Fix "'str' object does not support item assignment"
2017-03-26 21:51:09 +08:00
2bfaf89b6c [downloader/hls] move check for m3u8 live streams to get_suitable_downloader 2017-03-25 23:07:05 +01:00
4f06c1c9fc Merge branch 'master' of github.com-rndusr:rg3/youtube-dl into fix/str-item-assignment 2017-03-25 21:36:59 +01:00
942b44a052 [test_compat] Do not use dash in env variables' names 2017-03-26 03:24:25 +07:00
a426ef6d78 [test_utils] Do not use dash in env variables' names 2017-03-26 03:22:48 +07:00
41c5e60dd5 [test_utils] Fix expand_path tests 2017-03-26 03:07:56 +07:00
d212c93d16 [pluralsight] PEP 8 2017-03-26 02:34:25 +07:00
15495cf3e5 [franceculture] PEP 8 2017-03-26 02:32:46 +07:00
5b7cc56b05 [atresplayer] PEP 8 2017-03-26 02:32:14 +07:00
590bc6f6a1 Use expand_path where appropriate (closes #12556) 2017-03-26 02:31:16 +07:00
51098426b8 [utils] Introduce expand_path 2017-03-26 02:30:10 +07:00
c73e330e7a _find_jwplayer_data() returns dict or None
This simplifies code for callers of `_find_jwplayer_data()` which no longer have
to run `_parse_json()` on the return value.

It also makes sure that `_find_jwplayer_data()` returns either a `dict` or
`None` and nothing else.
2017-03-25 19:38:30 +01:00
fb4fc44928 [downloader/hls] immediately delegate downloading to ffmpeg in case live stream 2017-03-25 19:38:23 +01:00
03486dbb01 Add test for JWPlayer where config is passed as variable 2017-03-25 19:37:45 +01:00
51ef4919df [afreecatv] Fix extraction (closes #12179) 2017-03-26 01:32:07 +08:00
d66d43c554 [atvat] Add new extractor(closes #5325) 2017-03-25 18:13:58 +01:00
610a6d1053 [atresplayer] Do not extract ISM formats
As per @remitamine: the ISM downloader does not support videos served from wowza servers(it will produce broken files)
2017-03-25 21:40:54 +07:00
c6c22e984d [test_download] Print additional IEs in summary output 2017-03-25 22:36:40 +08:00
d97729c83a [fox] remove unused import 2017-03-25 14:28:53 +01:00
7aa0ee321b [fox] Add metadata extraction
Add series, season number, episode number and episode.
2017-03-25 21:12:25 +08:00
e8e4cc5a6a [generic] Replace LazyYT test with skiplagged
discourse.ubuntu.com has gone away, repalce with skiplagged.com.
Be nice to have a non-frontpage URL that might be more stable,
though I don't have one. Maybe this should move to html
in test/test_InfoExtractor.py?
2017-03-25 19:53:32 +07:00
c7301e677b [atresplayer] Extract DASH and ISM formats 2017-03-25 18:03:46 +07:00
048086920b [atresplayer] Extract HD manifest 2017-03-25 17:52:04 +07:00
1088d76da6 [atresplayer] Fix login error detection 2017-03-25 17:47:35 +07:00
31a1214076 [franceculture] fix extraction(closes #12547) 2017-03-25 07:04:48 +01:00
d0ba55871e [youtube] Improve _VALID_URLs (closes #12538) 2017-03-25 01:18:33 +07:00
54b960f340 [generic] Do not follow redirects to the same URL 2017-03-24 00:45:24 +07:00
a3ccd6bd11 release 2017.03.24 2017-03-24 00:24:23 +07:00
7963b6cba8 [ChangeLog] Actualize 2017-03-24 00:19:58 +07:00
bea7af6947 [channel9] Remove expired comment and sort imports 2017-03-23 23:58:12 +07:00
a5d783f525 [channel9] Extract more formats 2017-03-23 23:47:43 +07:00
d0572557c2 [ninecninemedia] remove mp4 url extraction request 2017-03-23 13:53:07 +01:00
52d5ecabd5 [bellmedia] add support for etalk.ca(closes #12447) 2017-03-23 13:52:45 +01:00
b0f7f21cb9 [channel9] fix extraction(closes #11323) 2017-03-23 09:22:37 +01:00
579c99a284 [cloudy] Fix extraction (closes #12525) 2017-03-22 23:48:06 +07:00
ca5ed022e9 [hbo] add support for free episode urls and new formats extraction(closes #12519) 2017-03-22 17:28:53 +01:00
391d076d7c [condenast] Fix extraction and style (closes #12526) 2017-03-22 23:22:14 +07:00
c183e14f89 [viu] Relax _VALID_URL (closes #12529) 2017-03-22 22:26:59 +07:00
093dad9e25 release 2017.03.22 2017-03-22 02:36:50 +07:00
e8686e51d7 [ChangeLog] Actualize 2017-03-22 02:35:09 +07:00
8e5a7c5e67 [pluralsight] Omit module title from video title (closes #12506) 2017-03-22 02:28:04 +07:00
e1e35d1ac6 [pornhub] Improve extraction and style (closes #12515) 2017-03-22 01:59:27 +07:00
21fbf0f955 [pornhub] Decode obfuscated video URL (closes #12470) 2017-03-22 01:51:45 +07:00
97952bdb78 [generic] Add test for Senate ISVP iframe embed 2017-03-22 01:12:14 +08:00
8a8cc339b6 [senateisvp] Allow https URL scheme for embeds 2017-03-20 23:35:13 +08:00
957f453429 [Openload.co] Fixed Extraction
They did it again. just a minor change though. here's quick fix
2017-03-20 16:15:00 +08:00
0e9a73e612 release 2017.03.20 2017-03-20 00:07:57 +07:00
0ecdd3adbd [ChangeLog] Actualize 2017-03-20 00:03:58 +07:00
9487ce03e9 [YoutubeDL] Allow multiple input URLs to be used with stdout as output template 2017-03-19 23:59:40 +07:00
45e6ad21b4 Credit @mrBliss for vtm (#11912) 2017-03-19 23:48:02 +07:00
68220649fa [ChangeLog] Update after #12099 2017-03-19 20:42:17 +08:00
46b18f2349 [BostonGlobe] New. Nonstandard version of Brightcove.
Has a "data-brightcove-video-id" instead of a "data-video-id," otherwise
pretty much just Brightcove. Except the Globe isn't all Brightcove
videos, so fallback to Generic, too.

Also, abstract playlist_from_matches() from generic.py to common.py, and use
it here.

History of these changes can be found in
51170427d4b1143572a498dedaee61863a5b2c5b.
2017-03-19 20:40:31 +08:00
772b5ff57f [toongoggles] Add new extractor(closes #12171) 2017-03-19 00:45:38 +01:00
f68ef1e2ab [medialaan] Remove unrelated test 2017-03-18 23:23:47 +07:00
febfe1e262 [adobepass] Detect and output error on authz token extraction (#12472) 2017-03-18 06:21:31 +07:00
5f0daab1ca [openload] Fix extraction 2017-03-18 07:02:55 +08:00
2a721cdff2 [medialaan] Fix and improve extraction (closes #11912) 2017-03-18 05:58:54 +07:00
e7a51a4c02 [vtm] Add extractor (closes #9974)
Implementation of the approach described in #9974.
2017-03-18 00:27:04 +07:00
3e5856d860 [discoverynetworks] add support for more domains and bypass geo restiction 2017-03-17 09:53:44 +01:00
ea883a687c [openload] Fix extraction (closes #10408)
Thanks to @makgun02

Ref: http://pastebin.com/raw/JX9gHFUz
2017-03-17 15:22:34 +08:00
7f3590c43b [test_InfoExtractor] Add some realworld tests for _extract_jwplayer_data 2017-03-17 00:00:01 +07:00
7d539ee10a release 2017.03.16 2017-03-16 22:42:12 +07:00
6ad476079d [ChangeLog] Actualize 2017-03-16 22:39:48 +07:00
0efbc6b56d [options] Mention flac support and sort alphabetically among the audio formats 2017-03-16 12:54:47 +01:00
21bfcd3d6e [postprocessor/ffmpeg] Add support for flac
Requested at http://stackoverflow.com/q/42828041/35070
2017-03-16 12:50:45 +01:00
b51dc9db0e [extractor/common] Extract SMIL formats from jwplayer 2017-03-16 03:30:53 +07:00
a309684285 [extractor/generic] Add forgotten return for jwplayer formats 2017-03-16 03:28:01 +07:00
ba448445b8 [redbull] improve extraction
- extract 1080p quality
- correct ttml subtitle ext
- catch api errors
- reduce request size
2017-03-15 01:40:54 +01:00
5db83d79bf release 2017.03.15 2017-03-15 02:01:24 +07:00
2a751e137f [ChangeLog] Actualize 2017-03-15 02:00:10 +07:00
398887b4c0 [Openload] Fixed Extraction
They did changed it again.
2017-03-14 14:03:52 +08:00
66bf351f80 [facebook] Make title optional (closes #12443) 2017-03-14 00:38:07 +07:00
9d08963022 [telecinco] Add test for #12430 2017-03-13 22:41:28 +07:00
e313d209c2 [mitele] Add support for ooyala videos (closes #12430) 2017-03-13 22:39:15 +07:00
ff9d509d20 [openload] Fix extraction
Just a minor fix for openload
2017-03-13 04:22:35 +08:00
c1795ca6c8 [streamable] Update API URL 2017-03-13 02:51:59 +08:00
8c99623259 [crunchyroll] Extract season name 2017-03-12 12:18:10 +08:00
57b0ddb35f [discoverygo] Actualize test 2017-03-11 23:21:08 +07:00
a28f8d7396 [discoverygo] Bypass geo restriction 2017-03-11 23:18:42 +07:00
7049799470 [discoverygo:playlist] Add extractor (closes #12424) 2017-03-11 23:16:51 +07:00
4605c94d1a [__init__] Fix missing subtitles if --add-metadata is used (#12423)
The previous fix for #5594 is incorrect
2017-03-11 19:37:45 +08:00
a8e687a4da release 2017.03.10 2017-03-10 23:26:28 +07:00
f9e5c92c94 [ChangeLog] Actualize 2017-03-10 23:23:24 +07:00
c2ee861c6d [extractor/generic] Make title optional for jwplayer embeds (closes #12410) 2017-03-10 23:16:53 +07:00
bd34c32bd7 [wdr] Actualize comment 2017-03-10 23:07:36 +07:00
f802c48660 [wdr:maus] Fix extraction and update tests 2017-03-10 23:59:32 +08:00
76bee08fe7 [prosiebensat1] Improve title extraction and add test 2017-03-09 23:42:07 +07:00
2913821723 [prosiebensat1] Improve title extraction (closes #12318) 2017-03-10 00:18:37 +08:00
0e7f9a9b48 [dplayit] Relax playback info URL extraction 2017-03-08 21:30:30 +07:00
0cf2352e85 [dplayit] Separate and rewrite extractor and bypass geo restriction (closes #12393) 2017-03-08 21:20:01 +07:00
0f6b87d067 [miomio] Fix extraction
Closes #12291
Closes #12388
Closes #12402
2017-03-08 19:46:58 +08:00
d7344d33b1 [telequebec] Fix description extraction and update test (closes #12399) 2017-03-08 18:25:59 +07:00
b08cc749d6 [openload] Fix extraction 2017-03-08 06:01:27 +08:00
b68a812ea8 [extractor/generic] Add test for brigthcove UUID-like videoPlayer 2017-03-07 23:00:21 +07:00
2e76bdc850 [brightcove:legacy] Relax videoPlayer validation check (closes #12381) 2017-03-07 22:59:33 +07:00
fe646a2f10 [twitch] PEP8 2017-03-07 15:34:06 +08:00
9df53ea36e Credit @puxlit for twitch 2fa (#11974) 2017-03-07 04:05:47 +07:00
d7d7f84c95 Credit @benages for redbull.tv (#11948) 2017-03-07 04:05:47 +07:00
dccd0ab35d release 2017.03.07 2017-03-07 03:59:22 +07:00
80146dcc6c [ChangeLog] Actualize 2017-03-07 03:57:54 +07:00
e30ccf7047 [soundcloud] Update client id (closes #12376) 2017-03-06 23:05:38 +07:00
54a3a8827b [__init__] Metadata should be added after conversion
Fixes #5594
2017-03-06 18:09:12 +08:00
92cb5763f4 [ChangeLog] Update after #12357 2017-03-06 18:04:19 +08:00
da92da4b88 Openload fix extraction (#12357)
* Fix extraction
2017-03-06 18:00:17 +08:00
1664702626 release 2017.03.06 2017-03-06 04:04:39 +07:00
3f116b189b [ChangeLog] Actualize 2017-03-06 04:01:21 +07:00
4b5de77bdb [utils] Process bytestrings in urljoin (closes #12369) 2017-03-06 03:57:46 +07:00
96182695e4 [drtv] Add geo countries to GeoRestrictedError 2017-03-06 03:23:42 +07:00
fc11ad3833 [drtv:live] Bypass geo restriction 2017-03-06 03:23:42 +07:00
d2b64e04b4 [addanime] Skip an invalid test 2017-03-06 00:35:04 +08:00
5dd376345b [tunepk] Add extractor (closes #12197, closes #12243) 2017-03-05 23:31:38 +07:00
1a2192cb90 [extractor/common] Pass arguments to _parse_jwplayer_formats and PEP8 2017-03-05 23:29:17 +07:00
0236cd0dfd [extractor/common] Improve height extraction and extract bitrate 2017-03-05 23:25:03 +07:00
ed0cf9b383 [extractor/common] Move jwplayer formats extraction in separate method 2017-03-05 23:22:27 +07:00
a50862b735 [downloader/external] Add missing import and PEP8 2017-03-05 10:24:29 +07:00
6d0fe752bf [external:ffmpeg] In test harness, limit to 10k download size
Otherwise, if you screw up a playlist test by including a playlist
dictionary key, you'll be there for eons while it downloads all the
files before erroring out.
2017-03-05 11:19:44 +08:00
afa4597618 release 2017.03.05 2017-03-05 02:23:08 +07:00
75027364ba [ChangeLog] Actualize 2017-03-05 02:22:02 +07:00
5316566edc [twitch] Use better naming and simplify (closes #11974) 2017-03-05 02:06:33 +07:00
c64c03be35 [twitch] Add basic support for two-factor authentication 2017-03-05 01:06:27 +07:00
bcefc59279 Credit @vierbergenlars for vijf.be (#12304) 2017-03-05 00:03:59 +07:00
6f211dc936 Credit @obilodeau for vrak (#11452) 2017-03-05 00:03:59 +07:00
f24c1e5584 Credit @TobiX for #9725 2017-03-05 00:03:59 +07:00
466274fe9a Credit @p2004a for vodpl (#12122) 2017-03-05 00:03:59 +07:00
30f8f142d4 Credit @ThomasChr for #12015 and #12245 2017-03-05 00:03:59 +07:00
a3ba8a7acf [vier] Add support for vijf.be
vier.be and vijf.be run on the same CMS and are property of the same company,
so the same extractor can be used for both of them.
2017-03-05 00:47:19 +08:00
054a587de8 [redbulltv] Improve extraction (closes #11948, closes #3919) 2017-03-04 23:28:21 +07:00
64b7ccef3e [redbulltv] Add extractor 2017-03-04 23:26:15 +07:00
6f4e4132d8 [douyutv] Switch to the PC API to escape the 5-min limitation
Thanks @spacemeowx2 for the algo.

Ref: https://gist.github.com/spacemeowx2/629b1d131bd7e240a7d28742048e80fc

Closes #12316
2017-03-04 23:23:18 +08:00
eb3079b6ce [generic] Add support for rutube embeds 2017-03-04 00:46:33 +07:00
bc82f22879 [rutube] Relax _VALID_URL 2017-03-04 00:42:51 +07:00
4d058c9862 [vrak] Improve and update test (closes #11452) 2017-03-03 23:58:16 +07:00
d16f27ca27 [brightcove:new] Add ability to smuggle geo_countries into URL 2017-03-03 23:58:03 +07:00
cbb127568a [vrak] Add extractor 2017-03-03 23:54:21 +07:00
d02d4fa0a9 [brightcove:new] Raise GeoRestrictedError 2017-03-03 22:49:48 +07:00
692fa200ca [go] Relax _VALID_URL (closes #12341) 2017-03-03 22:28:34 +07:00
9bae185ba6 [24video] Use original host for requests (closes #12339) 2017-03-03 22:16:00 +07:00
4d345bf17b [ruutu] Disable DASH formats (closes #12322)
Due to causing out of sync issue
2017-03-02 23:53:46 +07:00
250eea6821 release 2017.03.02 2017-03-02 22:33:22 +07:00
28d15b73f8 [ChangeLog] Actualize 2017-03-02 22:29:56 +07:00
11bb6ad1a5 [facebook] Fix extraction (closes #12323)
Almost all videos now use the pagelet type 'permalink_video_pagelet'
2017-03-02 20:51:24 +08:00
c9612c0487 [youtube] Mark errors about rental videos as expected
Closes #12324
2017-03-02 16:59:53 +08:00
af5049f128 [adobepass] Add Charter Spectrum (#11465)
Thanks @tv21 for the fix!
2017-03-02 02:15:51 +08:00
158af5242e [utils] Carry long doc string 2017-03-01 23:04:02 +07:00
40df485f55 [YoutubeDL] Don't sanitize identifiers (closes #12317) 2017-03-01 23:03:36 +07:00
4b8a984c67 [npo] Add support for audio 2017-03-01 22:21:13 +07:00
83e8fce628 [npo] Improve extraction and update tests 2017-03-01 22:14:46 +07:00
aa9cc2ecbf [npo] Adapt to app.php API (closes #12311) 2017-03-01 05:03:35 +07:00
1dc24093f8 release 2017.02.28 2017-02-28 23:59:22 +07:00
11bae9cdde [ChangeLog] Actualize 2017-02-28 23:49:24 +07:00
43b38424a9 [azmedien:showplaylist] Improve (closes #12160) 2017-02-28 23:37:54 +07:00
948519b35d [azmedien:showplaylist] Add support for all episodes playlists 2017-02-28 23:36:05 +07:00
87dadd456a [youtube:playlist] Recognize another playlist pattern (closes #11928, closes #12286) 2017-02-28 23:06:47 +07:00
7c4aa6fd6f [daisuki] Add subtitles (#4738) 2017-02-28 22:29:01 +08:00
9bd05b5a18 [daisuki] Add new extractor (closes #4738) 2017-02-28 22:19:26 +08:00
0a5445ddbe [utils] Add bytes_to_long() and long_to_bytes()
Used in daisuki.net (#4738)

Both are adapted from public domain PyCrypto:
https://github.com/dlitz/pycrypto/blob/master/lib/Crypto/Util/number.py
2017-02-28 22:10:31 +08:00
f48409c7ac [utils] Add pkcs1pad
Used in daisuki.net (#4738)
2017-02-28 22:10:31 +08:00
c9619f0a17 [aes] Add aes_cbc_encrypt
Used in daisuki.net (#4738)
2017-02-28 22:10:31 +08:00
f4c68ba372 [douyu] Fix extraction and update _TESTS
They've switched from flv to hls

Closes #12301
2017-02-28 21:41:03 +08:00
ef48a1175d release 2017.02.27 2017-02-27 23:26:07 +07:00
c6184bcf7b [ChangeLog] Actualize 2017-02-27 23:24:03 +07:00
18abb74376 [npo] Relax _VALID_URL for zapp.nl 2017-02-27 23:13:51 +07:00
dbc01fdb6f [hetklokhuis] Fix IE_NAME 2017-02-27 23:10:29 +07:00
f264c62334 [npo] Add support for zapp.nl 2017-02-27 23:10:00 +07:00
0dc5a86a32 [npo] Add support for hetklokhuis.nl (closes #12293) 2017-02-27 22:43:19 +07:00
0e879f432a [youtube:channel] Remove duplicate test 2017-02-27 22:22:43 +07:00
892b47ab6c [scivee] Remove extractor (#9315)
The Wikipedia page is changed from active to down:
https://en.wikipedia.org/w/index.php?title=SciVee&diff=prev&oldid=723161154

Some other interesting bits:

$ nslookup www.scivee.tv
Server:         8.8.8.8
Address:        8.8.8.8#53

Non-authoritative answer:
www.scivee.tv   canonical name = scivee.rcsb.org.
Name:   scivee.rcsb.org
Address: 132.249.231.211

$ nslookup rcsb.org
Server:         8.8.8.8
Address:        8.8.8.8#53

Non-authoritative answer:
Name:   rcsb.org
Address: 132.249.231.77

Both IPs are from UCSD. I guess it's maintained by a lab and they don't
maintain it anymore.
2017-02-27 21:34:33 +08:00
fdeea72611 [cda] Decode URL (fixes #12255) 2017-02-26 22:05:52 +08:00
xbe
7fd4655256 [crunchyroll] Extract uploader name that's not a link
Provide the Crunchyroll extractor with the ability to extract uploader
names that aren't links. Add a test for this new functionality.
This fixes #12267.
2017-02-26 19:08:10 +08:00
fd5c4aab59 [youtube] Raise GeoRestrictedError 2017-02-26 16:52:40 +07:00
8878789f11 [dailymotion] Raise GeoRestrictedError 2017-02-26 16:52:40 +07:00
a5cf17989b [MDR] Relax _VALID_URL and playerURL matching and update _TESTS
Ref: #12169
2017-02-26 17:24:54 +08:00
b3aec47665 [tvigle] Raise GeoRestrictedError 2017-02-25 23:27:45 +07:00
9d0c08a02c [vevo] Fix videos with the new streams/streamsV3 format (closes #11719) 2017-02-26 00:15:49 +08:00
e498758b9c [freshlive] Fix issues and improve (closes #12175) 2017-02-25 22:56:42 +07:00
5fc8d89361 [freshlive] Add extractor 2017-02-25 22:55:17 +07:00
d374d943f3 [downloader/common] Limit displaying 2 digits after decimal point in sleep interval message 2017-02-25 20:59:04 +07:00
103f8c8d36 [xhamster] Capture and output videoClosed error (#12263) 2017-02-25 20:38:21 +07:00
922ab7840b [etonline] Add extractor (closes #12236) 2017-02-25 20:16:40 +07:00
831217291a [compat] Use try except for compat_numeric_types 2017-02-25 19:44:50 +07:00
db182c63fb [njpwworld] Add new extractor (closes #11561) 2017-02-25 18:44:39 +08:00
eeb0a95684 [extractor/common] Add 'preference' to _parse_html5_media_entries
Some websites, like NJPWorld, put different qualities on different
player pages.
2017-02-25 18:40:05 +08:00
231bcd0b6b [amcnetworks] Relax _VALID_URL (#12127) 2017-02-25 02:51:53 +07:00
204efc8509 release 2017.02.24.1 2017-02-24 21:59:39 +07:00
5d3a51e1b9 [ChangeLog] Actualize 2017-02-24 21:57:39 +07:00
ad3033037c [noco] Modernize 2017-02-24 21:51:56 +07:00
f3bc281239 [noco] Swtich login URL to https (closes #12246) 2017-02-24 21:48:34 +07:00
441d7a32e5 [thescene] Extract more metadata 2017-02-24 21:22:29 +07:00
51ed496307 [thescene] Fix extraction (closes #12235) 2017-02-24 22:08:45 +08:00
68f17a9c2d [tubitv] use geo bypass mechanism 2017-02-24 12:27:56 +01:00
39e7277ed1 [openload] fix extraction(closes #10408) 2017-02-24 11:21:58 +01:00
42dcdbe11c [ivi] Raise GeoRestrictedError 2017-02-24 10:54:39 +07:00
6b097cff27 release 2017.02.24 2017-02-24 06:09:15 +07:00
f2f7961820 [ChangeLog] Actualize 2017-02-24 06:07:41 +07:00
be5df5ee31 Suppress help for all deprecated options and print warning when used 2017-02-24 06:04:27 +07:00
f2980fddeb [lynda:course] Add webpage extraction fallback (closes #12238) 2017-02-24 05:01:31 +07:00
0f57447de7 [postprocessor/ffmpeg] Add mising space (closes #12232) 2017-02-24 04:56:58 +07:00
19f3821821 [devscripts/make_lazy_extractors] Fix making lazy extractors on python 3 under Windows 2017-02-24 02:09:51 +07:00
8e1409fd80 [go] sign all uplynk urls and use geo bypass only for free videos(closes #12087)(closes #12210) 2017-02-23 18:42:06 +01:00
050f143c12 [README.md] Clarify sequence types in output template and document numeric string formatting operations 2017-02-23 23:00:13 +07:00
fafc2bf5a9 [options] Deprecate --autonumber-size 2017-02-23 22:11:16 +07:00
b3175982c3 [YoutubeDL] Add more numeric fields for NA substitution in outtmpl 2017-02-23 22:01:57 +07:00
89db639dfe [YoutubeDL] Rewrite outtmpl for playlist_index and autonumber for backward compatibility 2017-02-23 22:01:09 +07:00
d0d9ade486 [YoutubeDL] Add support for string formatting operations in output template 2017-02-23 22:57:53 +08:00
28572a1a0b [compat] Add compat_numeric_types 2017-02-23 22:57:53 +08:00
0f3d41b44d [devscripts/run_tests] Exclude youtube lists tests from core build 2017-02-23 19:48:54 +07:00
d5fd9a3be3 [skylinewebcams] Add extractor (closes #12221) 2017-02-23 18:45:38 +07:00
ada77fa544 [instagram] Add support for multi video posts (closes #12226) 2017-02-23 18:02:04 +07:00
9e03aa75c7 [crunchyroll] extract playlist entries ids 2017-02-23 11:57:18 +01:00
30eaa3a702 [mgtv] fix extraction 2017-02-23 11:57:05 +01:00
c59f703610 [sohu] raise GeoRestrictedError 2017-02-23 11:56:55 +01:00
bc61c80c14 [leeco] raise GeoRestrictedError and use geo bypass mechanism 2017-02-23 11:56:45 +01:00
345b24538b release 2017.02.22 2017-02-22 23:50:42 +07:00
63a29b6118 [ChangeLog] Actualize 2017-02-22 23:45:01 +07:00
b5869560a4 [crunchyroll] Fix descriptions with double quotes (closes #12124) 2017-02-23 00:08:45 +08:00
527ef85fe9 [dailymotion] Make comment count optional (closes #12209)
Not served anymore
2017-02-22 21:49:30 +07:00
58ad6995cd [vidzi] Add test for #12213 2017-02-22 21:29:53 +07:00
a86e416088 [vidzi] Add support for vidzi.cc 2017-02-22 22:28:09 +08:00
71e9577b94 [24video] Add support for 24video.tube (closes #12217) 2017-02-22 21:19:52 +07:00
0d427c8304 [setup] Actualize maintainer info 2017-02-22 01:51:27 +07:00
139d8ac106 [setup] Add python 3.6 classifier 2017-02-22 01:50:34 +07:00
abd29a2ced [crackle] use geo bypass mechanism 2017-02-21 19:37:26 +01:00
31615ac279 [viewster] use geo verifcation headers 2017-02-21 19:36:39 +01:00
fc320a40d9 Revert "[cbc] use geo bypass mechanism"
This reverts commit 86466a8b6f.
2017-02-21 18:14:55 +01:00
7345d6d465 [tfo] Improve geo restriction detection and use geo bypass mechanism 2017-02-21 17:52:50 +01:00
86466a8b6f [cbc] use geo bypass mechanism 2017-02-21 17:52:50 +01:00
33dc173cdc [telequebec] use geo bypass mechanism 2017-02-21 17:52:50 +01:00
3444844b04 [limelight] extract PlaylistService errors 2017-02-21 17:52:50 +01:00
8c6c88c7da release 2017.02.21 2017-02-21 23:48:24 +07:00
159aaaa9d0 [ChangeLog] Actualize 2017-02-21 23:46:58 +07:00
eea0716cae [extractor/common] Print origin country for fake IP 2017-02-21 23:14:33 +07:00
336a76551b [extractor/common] Do not quit _initialize_geo_bypass on empty countries 2017-02-21 23:09:41 +07:00
dc0a869e5e [extractor/common] Fix typo 2017-02-21 23:05:31 +07:00
e39b5d4ab8 [extractor/common] Allow calling _initialize_geo_bypass from extractors (#11970) 2017-02-21 23:00:43 +07:00
e469ab2528 [ninecninemedia] use geo bypass mechanism 2017-02-21 14:38:00 +01:00
890d44b005 [adobepass] add support for Time Warner Cable(closes #12191) 2017-02-20 19:00:40 +01:00
6926304472 [spankbang] Make uploader optional (closes #12193) 2017-02-21 00:54:43 +07:00
3ccdde8cb7 [extractor/common] Emphasize geo bypass APIs are experimental 2017-02-20 23:21:15 +07:00
da42ff0668 [iprima] Improve geo restriction detection and disable geo bypass 2017-02-20 23:17:19 +07:00
82f662182b [iprima] Modernize 2017-02-20 23:16:14 +07:00
2cc7fcd338 [commonmistakes] Disable UnicodeBOM extractor test for python 3.2 2017-02-20 03:06:52 +07:00
6d4c259765 [svt] PEP 8 2017-02-20 02:25:55 +07:00
c78dd35491 [nrk] PEP 8 2017-02-20 02:25:39 +07:00
8ffb8e63fe [prosiebensat1] Throw ExtractionError on unsupported page type (closes #12180) 2017-02-20 01:00:53 +07:00
983e9b7746 [nrk] Update _API_HOST and relax _VALID_URL 2017-02-20 00:59:31 +07:00
8936f68a0b [travis] Run tests in parallel
[test_download] Print test names in case of network errors

[test_download] Add comments for nose parameters

[test_download] Modify outtmpl to prevent info JSON filename conflicts

Thanks @jaimeMF for the idea.

[travis] Only download tests should be run in parallel
2017-02-19 21:26:35 +08:00
c58b7ffef4 [tv4] Bypass geo restriction and improve detection 2017-02-19 06:25:59 +07:00
f1a78ee4ef [tv4] Switch to hls3 protocol (closes #12177) 2017-02-19 06:16:00 +07:00
de64e23c56 [downloader/ism] Honor HTTP headers when downloading fragments 2017-02-19 04:18:36 +07:00
553f6dbac7 [downloader/dash] Honor HTTP headers when downloading fragments
For example, https://www.oppetarkiv.se/video/1196142/natten-ar-dagens-mor
2017-02-19 04:18:22 +07:00
0aa10994f4 [options] Move geo restriction related options to separate section 2017-02-19 05:10:08 +08:00
4248dad92b Improve geo bypass mechanism
* Rename options to preffixly match with --geo-verification-proxy
* Introduce _GEO_COUNTRIES for extractors
* Implement faking IP right away for sites with known geo restriction
2017-02-19 05:10:08 +08:00
0a840f584c Rename bypass geo restriction options 2017-02-19 05:10:08 +08:00
0016b84e16 Add faked X-Forwarded-For to formats' HTTP headers 2017-02-19 05:10:08 +08:00
18a0defab0 [utils] Make random_ipv4 return unicode string 2017-02-19 05:10:08 +08:00
5d3fbf77d9 [viki] Improve geo restriction detection 2017-02-19 05:10:08 +08:00
80b59020e0 [vgtv] Improve geo restriction detection 2017-02-19 05:10:08 +08:00
71631862f4 [srgssr] Improve geo restriction detection 2017-02-19 05:10:08 +08:00
89cc7fe770 [vbox7] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
04d906eae3 [svt] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
8ab8066cf0 [pbs] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
01b1aa9ff4 [ondemandkorea] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
ff4007891f [nrk] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
28200e654b [itv] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
e633f21a96 [go] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
d392005a79 [dramafever] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
773f291dcb Add experimental geo restriction bypass mechanism
Based on faking X-Forwarded-For HTTP header
2017-02-19 05:10:08 +08:00
bf5b9d859a [utils] Introduce YoutubeDLError base class for all youtube-dl exceptions 2017-02-19 05:10:08 +08:00
049a0f4d6d [brightcove:legacy] restrict videoPlayer value(closes #12040) 2017-02-18 21:08:40 +01:00
ac33accd96 [options] Mention quoted string literals for --match-filter 2017-02-18 23:59:26 +07:00
e84888b432 [tvn24] Improve extraction (closes #11679) 2017-02-18 23:34:09 +07:00
02d9b82a23 [tvn24] Add extractor 2017-02-18 23:33:49 +07:00
a2e3286676 [thisav] Add support for html5 media (closes #11771) 2017-02-18 20:21:53 +07:00
f75caf059e [metacafe] Improve (closes #10371) 2017-02-18 19:58:25 +07:00
bdabbc220c [metacafe] Bypass family filter
If you don't send this user=ffilter: false cookie, it will 301 redirect you to a page asking about it, and then the title check will fail.
2017-02-18 19:47:33 +07:00
70bcc444a9 [viceland] improve info extraction and update test 2017-02-18 09:52:43 +01:00
28e35f5070 release 2017.02.17 2017-02-17 23:59:56 +07:00
cf3704c132 [ChangeLog] Actualize 2017-02-17 23:48:30 +07:00
2c1f442c2b [options] Add missing spaces 2017-02-17 23:18:26 +07:00
bad4ccdb5d [heise] Improve (closes #9725) 2017-02-17 23:09:40 +07:00
db76c30c6e [heise] Support videos embedded in any article. 2017-02-17 22:55:53 +07:00
c2bde5d081 [ellentv] Improve 2017-02-17 22:45:51 +07:00
90fad0e74c [openload] Fix extraction (closes #12002) 2017-02-17 22:31:16 +07:00
d94badc755 [openload] Semifix extraction (closes #10408)
just updated the code. i don't do much python still i tried to convert my code. lemme know if there is any prob with it
2017-02-17 22:30:05 +07:00
fef51645d6 [theplatform] Recognize URLs with whitespaces (closes #12044) 2017-02-17 23:13:51 +08:00
4cead6a614 [einthusan] Relax _VALID_URL (closes #12141, closes #12159) 2017-02-17 22:02:01 +07:00
a4a554a793 [generic] Try parsing JWPlayer embedded videos (closes #12030) 2017-02-16 23:44:03 +08:00
b898f0a173 [elpais] Fix typo and improve extraction (closes #12139) 2017-02-16 04:57:42 +07:00
2480b056c1 release 2017.02.16 2017-02-16 00:10:04 +07:00
3aa25395aa [ChangeLog] Actualize 2017-02-16 00:08:56 +07:00
eafaeb226a [ceskatelevize] Lower priority for audio description sources (#12119) 2017-02-16 00:04:15 +07:00
de4d378c0c [ceskatelevize] Prefix format ids 2017-02-15 23:38:00 +07:00
099cfdb770 [devscripts/run_tests.sh] Change permission for script to 755 2017-02-16 00:28:31 +08:00
398dea3210 [test_YoutubeDL] Fix invalid escape sequences 2017-02-15 23:20:46 +07:00
db13c16ef8 [utils] Add support for quoted string literals in --match-filter (closes #8050, closes #12142, closes #12144) 2017-02-15 23:12:10 +07:00
1bd05345ea [amcnetworks] fix extraction(closes #12127) 2017-02-15 14:19:18 +01:00
3021cf83b7 [pinkbike] Fix uploader extraction (closes #12054) 2017-02-15 02:08:32 +07:00
04a741232f [onetpl] Add support for businessinsider.com.pl and plejada.pl 2017-02-15 01:23:55 +07:00
43a3d9edfc [onetpl] Add support for onet.pl (closes #10507) 2017-02-15 01:14:06 +07:00
d31aa74fdb [onetmvp] Add shortcut extractor 2017-02-15 00:58:18 +07:00
6092ccd058 [vodpl] Make more robust and add another test (closes #12122) 2017-02-15 00:52:31 +07:00
22ce9ad2bd [vod.pl] Add new extractor 2017-02-15 00:48:08 +07:00
9a372f14b4 [pornhub] Extract video URL from tv platform site (#12007, #12129) 2017-02-14 23:52:41 +07:00
5cb2d36c82 [ceskatelevize] Extract DASH formats (closes #12119, closes #12133) 2017-02-14 22:57:38 +07:00
fcca0d53a8 [ceskatelevize] Quick fix to revert to using old HLS-based playlist
This fixes recent changes in iVysilani. Proper patch should migrate to
MPEG-DASH version, which is now the default.
2017-02-14 22:25:37 +07:00
744 changed files with 63572 additions and 26200 deletions

View File

@ -1,60 +0,0 @@
## Please follow the guide below
- You will be asked some questions and requested to provide some information, please read them **carefully** and answer honestly
- Put an `x` into all the boxes [ ] relevant to your *issue* (like that [x])
- Use *Preview* tab to see how your issue will actually look like
---
### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2017.02.14*. If it's not read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
- [ ] I've **verified** and **I assure** that I'm running youtube-dl **2017.02.14**
### Before submitting an *issue* make sure you have:
- [ ] At least skimmed through [README](https://github.com/rg3/youtube-dl/blob/master/README.md) and **most notably** [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections
- [ ] [Searched](https://github.com/rg3/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones
### What is the purpose of your *issue*?
- [ ] Bug report (encountered problems with youtube-dl)
- [ ] Site support request (request for adding support for a new site)
- [ ] Feature request (request for a new functionality)
- [ ] Question
- [ ] Other
---
### The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your *issue*
---
### If the purpose of this *issue* is a *bug report*, *site support request* or you are not completely sure provide the full verbose output as follows:
Add `-v` flag to **your command line** you run youtube-dl with, copy the **whole** output and insert it here. It should look similar to one below (replace it with **your** log inserted between triple ```):
```
$ youtube-dl -v <your command line>
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version 2017.02.14
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
[debug] Proxy map: {}
...
<end of log>
```
---
### If the purpose of this *issue* is a *site support request* please provide all kinds of example URLs support for which should be included (replace following example URLs by **yours**):
- Single video: https://www.youtube.com/watch?v=BaW_jenozKc
- Single video: https://youtu.be/BaW_jenozKc
- Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc
Note that **youtube-dl does not support sites dedicated to [copyright infringement](https://github.com/rg3/youtube-dl#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free)**. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
---
### Description of your *issue*, suggested solution and other information
Explanation of your *issue* in arbitrary form goes here. Please make sure the [description is worded well enough to be understood](https://github.com/rg3/youtube-dl#is-the-description-of-the-issue-itself-sufficient). Provide as much context and examples as possible.
If work on your *issue* requires account credentials please provide them or explain how one can obtain them.

63
.github/ISSUE_TEMPLATE/1_broken_site.md vendored Normal file
View File

@ -0,0 +1,63 @@
---
name: Broken site support
about: Report broken or misfunctioning site
title: ''
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2020.12.02. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in http://yt-dl.org/escape.
- Search the bugtracker for similar issues: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a broken site support
- [ ] I've verified that I'm running youtube-dl version **2020.12.02**
- [ ] I've checked that all provided URLs are alive and playable in a browser
- [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
- [ ] I've searched the bugtracker for similar issues including closed ones
## Verbose log
<!--
Provide the complete verbose output of youtube-dl that clearly demonstrates the problem.
Add the `-v` flag to your command line you run youtube-dl with (`youtube-dl -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version 2020.12.02
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
[debug] Proxy map: {}
<more lines>
-->
```
PASTE VERBOSE LOG HERE
```
## Description
<!--
Provide an explanation of your issue in an arbitrary form. Provide any additional information, suggested solution and as much context and examples as possible.
If work on your issue requires account credentials please provide them or explain how one can obtain them.
-->
WRITE DESCRIPTION HERE

View File

@ -0,0 +1,54 @@
---
name: Site support request
about: Request support for a new site
title: ''
labels: 'site-support-request'
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2020.12.02. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
- Make sure that site you are requesting is not dedicated to copyright infringement, see https://yt-dl.org/copyright-infringement. youtube-dl does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
- Search the bugtracker for similar site support requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a new site support request
- [ ] I've verified that I'm running youtube-dl version **2020.12.02**
- [ ] I've checked that all provided URLs are alive and playable in a browser
- [ ] I've checked that none of provided URLs violate any copyrights
- [ ] I've searched the bugtracker for similar site support requests including closed ones
## Example URLs
<!--
Provide all kinds of example URLs support for which should be included. Replace following example URLs by yours.
-->
- Single video: https://www.youtube.com/watch?v=BaW_jenozKc
- Single video: https://youtu.be/BaW_jenozKc
- Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc
## Description
<!--
Provide any additional information.
If work on your issue requires account credentials please provide them or explain how one can obtain them.
-->
WRITE DESCRIPTION HERE

View File

@ -0,0 +1,37 @@
---
name: Site feature request
about: Request a new functionality for a site
title: ''
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2020.12.02. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Search the bugtracker for similar site feature requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a site feature request
- [ ] I've verified that I'm running youtube-dl version **2020.12.02**
- [ ] I've searched the bugtracker for similar site feature requests including closed ones
## Description
<!--
Provide an explanation of your site feature request in an arbitrary form. Please make sure the description is worded well enough to be understood, see https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient. Provide any additional information, suggested solution and as much context and examples as possible.
-->
WRITE DESCRIPTION HERE

65
.github/ISSUE_TEMPLATE/4_bug_report.md vendored Normal file
View File

@ -0,0 +1,65 @@
---
name: Bug report
about: Report a bug unrelated to any particular site or extractor
title: ''
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2020.12.02. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in http://yt-dl.org/escape.
- Search the bugtracker for similar issues: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Read bugs section in FAQ: http://yt-dl.org/reporting
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a broken site support issue
- [ ] I've verified that I'm running youtube-dl version **2020.12.02**
- [ ] I've checked that all provided URLs are alive and playable in a browser
- [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
- [ ] I've searched the bugtracker for similar bug reports including closed ones
- [ ] I've read bugs section in FAQ
## Verbose log
<!--
Provide the complete verbose output of youtube-dl that clearly demonstrates the problem.
Add the `-v` flag to your command line you run youtube-dl with (`youtube-dl -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version 2020.12.02
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
[debug] Proxy map: {}
<more lines>
-->
```
PASTE VERBOSE LOG HERE
```
## Description
<!--
Provide an explanation of your issue in an arbitrary form. Please make sure the description is worded well enough to be understood, see https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient. Provide any additional information, suggested solution and as much context and examples as possible.
If work on your issue requires account credentials please provide them or explain how one can obtain them.
-->
WRITE DESCRIPTION HERE

View File

@ -0,0 +1,38 @@
---
name: Feature request
about: Request a new functionality unrelated to any particular site or extractor
title: ''
labels: 'request'
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2020.12.02. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Search the bugtracker for similar feature requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a feature request
- [ ] I've verified that I'm running youtube-dl version **2020.12.02**
- [ ] I've searched the bugtracker for similar feature requests including closed ones
## Description
<!--
Provide an explanation of your issue in an arbitrary form. Please make sure the description is worded well enough to be understood, see https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient. Provide any additional information, suggested solution and as much context and examples as possible.
-->
WRITE DESCRIPTION HERE

38
.github/ISSUE_TEMPLATE/6_question.md vendored Normal file
View File

@ -0,0 +1,38 @@
---
name: Ask question
about: Ask youtube-dl related question
title: ''
labels: 'question'
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- Look through the README (http://yt-dl.org/readme) and FAQ (http://yt-dl.org/faq) for similar questions
- Search the bugtracker for similar questions: http://yt-dl.org/search-issues
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm asking a question
- [ ] I've looked through the README and FAQ for similar questions
- [ ] I've searched the bugtracker for similar questions including closed ones
## Question
<!--
Ask your question in an arbitrary form. Please make sure it's worded well enough to be understood, see https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient.
-->
WRITE QUESTION HERE

View File

@ -1,60 +0,0 @@
## Please follow the guide below
- You will be asked some questions and requested to provide some information, please read them **carefully** and answer honestly
- Put an `x` into all the boxes [ ] relevant to your *issue* (like that [x])
- Use *Preview* tab to see how your issue will actually look like
---
### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *%(version)s*. If it's not read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
- [ ] I've **verified** and **I assure** that I'm running youtube-dl **%(version)s**
### Before submitting an *issue* make sure you have:
- [ ] At least skimmed through [README](https://github.com/rg3/youtube-dl/blob/master/README.md) and **most notably** [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections
- [ ] [Searched](https://github.com/rg3/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones
### What is the purpose of your *issue*?
- [ ] Bug report (encountered problems with youtube-dl)
- [ ] Site support request (request for adding support for a new site)
- [ ] Feature request (request for a new functionality)
- [ ] Question
- [ ] Other
---
### The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your *issue*
---
### If the purpose of this *issue* is a *bug report*, *site support request* or you are not completely sure provide the full verbose output as follows:
Add `-v` flag to **your command line** you run youtube-dl with, copy the **whole** output and insert it here. It should look similar to one below (replace it with **your** log inserted between triple ```):
```
$ youtube-dl -v <your command line>
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version %(version)s
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
[debug] Proxy map: {}
...
<end of log>
```
---
### If the purpose of this *issue* is a *site support request* please provide all kinds of example URLs support for which should be included (replace following example URLs by **yours**):
- Single video: https://www.youtube.com/watch?v=BaW_jenozKc
- Single video: https://youtu.be/BaW_jenozKc
- Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc
Note that **youtube-dl does not support sites dedicated to [copyright infringement](https://github.com/rg3/youtube-dl#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free)**. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
---
### Description of your *issue*, suggested solution and other information
Explanation of your *issue* in arbitrary form goes here. Please make sure the [description is worded well enough to be understood](https://github.com/rg3/youtube-dl#is-the-description-of-the-issue-itself-sufficient). Provide as much context and examples as possible.
If work on your *issue* requires account credentials please provide them or explain how one can obtain them.

View File

@ -0,0 +1,63 @@
---
name: Broken site support
about: Report broken or misfunctioning site
title: ''
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is %(version)s. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in http://yt-dl.org/escape.
- Search the bugtracker for similar issues: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a broken site support
- [ ] I've verified that I'm running youtube-dl version **%(version)s**
- [ ] I've checked that all provided URLs are alive and playable in a browser
- [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
- [ ] I've searched the bugtracker for similar issues including closed ones
## Verbose log
<!--
Provide the complete verbose output of youtube-dl that clearly demonstrates the problem.
Add the `-v` flag to your command line you run youtube-dl with (`youtube-dl -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version %(version)s
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
[debug] Proxy map: {}
<more lines>
-->
```
PASTE VERBOSE LOG HERE
```
## Description
<!--
Provide an explanation of your issue in an arbitrary form. Provide any additional information, suggested solution and as much context and examples as possible.
If work on your issue requires account credentials please provide them or explain how one can obtain them.
-->
WRITE DESCRIPTION HERE

View File

@ -0,0 +1,54 @@
---
name: Site support request
about: Request support for a new site
title: ''
labels: 'site-support-request'
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is %(version)s. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
- Make sure that site you are requesting is not dedicated to copyright infringement, see https://yt-dl.org/copyright-infringement. youtube-dl does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
- Search the bugtracker for similar site support requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a new site support request
- [ ] I've verified that I'm running youtube-dl version **%(version)s**
- [ ] I've checked that all provided URLs are alive and playable in a browser
- [ ] I've checked that none of provided URLs violate any copyrights
- [ ] I've searched the bugtracker for similar site support requests including closed ones
## Example URLs
<!--
Provide all kinds of example URLs support for which should be included. Replace following example URLs by yours.
-->
- Single video: https://www.youtube.com/watch?v=BaW_jenozKc
- Single video: https://youtu.be/BaW_jenozKc
- Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc
## Description
<!--
Provide any additional information.
If work on your issue requires account credentials please provide them or explain how one can obtain them.
-->
WRITE DESCRIPTION HERE

View File

@ -0,0 +1,37 @@
---
name: Site feature request
about: Request a new functionality for a site
title: ''
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is %(version)s. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Search the bugtracker for similar site feature requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a site feature request
- [ ] I've verified that I'm running youtube-dl version **%(version)s**
- [ ] I've searched the bugtracker for similar site feature requests including closed ones
## Description
<!--
Provide an explanation of your site feature request in an arbitrary form. Please make sure the description is worded well enough to be understood, see https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient. Provide any additional information, suggested solution and as much context and examples as possible.
-->
WRITE DESCRIPTION HERE

View File

@ -0,0 +1,65 @@
---
name: Bug report
about: Report a bug unrelated to any particular site or extractor
title: ''
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is %(version)s. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in http://yt-dl.org/escape.
- Search the bugtracker for similar issues: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Read bugs section in FAQ: http://yt-dl.org/reporting
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a broken site support issue
- [ ] I've verified that I'm running youtube-dl version **%(version)s**
- [ ] I've checked that all provided URLs are alive and playable in a browser
- [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
- [ ] I've searched the bugtracker for similar bug reports including closed ones
- [ ] I've read bugs section in FAQ
## Verbose log
<!--
Provide the complete verbose output of youtube-dl that clearly demonstrates the problem.
Add the `-v` flag to your command line you run youtube-dl with (`youtube-dl -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version %(version)s
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
[debug] Proxy map: {}
<more lines>
-->
```
PASTE VERBOSE LOG HERE
```
## Description
<!--
Provide an explanation of your issue in an arbitrary form. Please make sure the description is worded well enough to be understood, see https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient. Provide any additional information, suggested solution and as much context and examples as possible.
If work on your issue requires account credentials please provide them or explain how one can obtain them.
-->
WRITE DESCRIPTION HERE

View File

@ -0,0 +1,38 @@
---
name: Feature request
about: Request a new functionality unrelated to any particular site or extractor
title: ''
labels: 'request'
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is %(version)s. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Search the bugtracker for similar feature requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a feature request
- [ ] I've verified that I'm running youtube-dl version **%(version)s**
- [ ] I've searched the bugtracker for similar feature requests including closed ones
## Description
<!--
Provide an explanation of your issue in an arbitrary form. Please make sure the description is worded well enough to be understood, see https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient. Provide any additional information, suggested solution and as much context and examples as possible.
-->
WRITE DESCRIPTION HERE

View File

@ -7,8 +7,9 @@
---
### Before submitting a *pull request* make sure you have:
- [ ] At least skimmed through [adding new extractor tutorial](https://github.com/rg3/youtube-dl#adding-support-for-a-new-site) and [youtube-dl coding conventions](https://github.com/rg3/youtube-dl#youtube-dl-coding-conventions) sections
- [ ] [Searched](https://github.com/rg3/youtube-dl/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
- [ ] At least skimmed through [adding new extractor tutorial](https://github.com/ytdl-org/youtube-dl#adding-support-for-a-new-site) and [youtube-dl coding conventions](https://github.com/ytdl-org/youtube-dl#youtube-dl-coding-conventions) sections
- [ ] [Searched](https://github.com/ytdl-org/youtube-dl/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
- [ ] Checked the code with [flake8](https://pypi.python.org/pypi/flake8)
### In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check one of the following options:
- [ ] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/)

7
.gitignore vendored
View File

@ -22,6 +22,7 @@ cover/
updates_key.pem
*.egg-info
*.srt
*.ttml
*.sbv
*.vtt
*.flv
@ -35,8 +36,8 @@ updates_key.pem
*.mkv
*.swf
*.part
*.ytdl
*.swp
test/testdata
test/local_parameters.json
.tox
youtube-dl.zsh
@ -46,3 +47,7 @@ youtube-dl.zsh
*.iml
tmp/
venv/
# VS Code related files
.vscode

View File

@ -7,18 +7,44 @@ python:
- "3.4"
- "3.5"
- "3.6"
sudo: false
- "pypy"
- "pypy3"
dist: trusty
env:
- YTDL_TEST_SET=core
- YTDL_TEST_SET=download
before_script:
- chmod +x ./devscripts/run_tests.sh
jobs:
include:
- python: 3.7
dist: xenial
env: YTDL_TEST_SET=core
- python: 3.7
dist: xenial
env: YTDL_TEST_SET=download
- python: 3.8
dist: xenial
env: YTDL_TEST_SET=core
- python: 3.8
dist: xenial
env: YTDL_TEST_SET=download
- python: 3.8-dev
dist: xenial
env: YTDL_TEST_SET=core
- python: 3.8-dev
dist: xenial
env: YTDL_TEST_SET=download
- env: JYTHON=true; YTDL_TEST_SET=core
- env: JYTHON=true; YTDL_TEST_SET=download
- name: flake8
python: 3.8
dist: xenial
install: pip install flake8
script: flake8 .
fast_finish: true
allow_failures:
- env: YTDL_TEST_SET=download
- env: JYTHON=true; YTDL_TEST_SET=core
- env: JYTHON=true; YTDL_TEST_SET=download
before_install:
- if [ "$JYTHON" == "true" ]; then ./devscripts/install_jython.sh; export PATH="$HOME/jython/bin:$PATH"; fi
script: ./devscripts/run_tests.sh
notifications:
email:
- filippo.valsorda@gmail.com
- yasoob.khld@gmail.com
# irc:
# channels:
# - "irc.freenode.org#youtube-dl"
# skip_join: true

44
AUTHORS
View File

@ -202,3 +202,47 @@ Fabian Stahl
Bagira
Odd Stråbø
Philip Herzog
Thomas Christlieb
Marek Rusinowski
Tobias Gruetzmacher
Olivier Bilodeau
Lars Vierbergen
Juanjo Benages
Xiao Di Guan
Thomas Winant
Daniel Twardowski
Jeremie Jarosh
Gerard Rovira
Marvin Ewald
Frédéric Bournival
Timendum
gritstub
Adam Voss
Mike Fährmann
Jan Kundrát
Giuseppe Fabiano
Örn Guðjónsson
Parmjit Virk
Genki Sky
Ľuboš Katrinec
Corey Nicholson
Ashutosh Chaudhary
John Dong
Tatsuyuki Ishi
Daniel Weber
Kay Bouché
Yang Hongbo
Lei Wang
Petr Novák
Leonardo Taccari
Martin Weinelt
Surya Oktafendri
TingPing
Alexandre Macabies
Bastian de Groot
Niklas Haas
András Veres-Szentkirályi
Enes Solak
Nathan Rossi
Thomas van der Berg
Luca Cherubin

View File

@ -3,7 +3,7 @@
$ youtube-dl -v <your command line>
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version 2015.12.06
[debug] Git HEAD: 135392e
@ -34,7 +34,7 @@ For bug reports, this means that your report should contain the *complete* outpu
If your server has multiple IPs or you suspect censorship, adding `--call-home` may be a good idea to get more diagnostics. If the error is `ERROR: Unable to extract ...` and you cannot reproduce it from multiple countries, add `--dump-pages` (warning: this will yield a rather large output, redirect it to the file `log.txt` by adding `>log.txt 2>&1` to your command-line) or upload the `.dump` files you get when you add `--write-pages` [somewhere](https://gist.github.com/).
**Site support requests must contain an example URL**. An example URL is a URL you might want to download, like `http://www.youtube.com/watch?v=BaW_jenozKc`. There should be an obvious video present. Except under very special circumstances, the main page of a video service (e.g. `http://www.youtube.com/`) is *not* an example URL.
**Site support requests must contain an example URL**. An example URL is a URL you might want to download, like `https://www.youtube.com/watch?v=BaW_jenozKc`. There should be an obvious video present. Except under very special circumstances, the main page of a video service (e.g. `https://www.youtube.com/`) is *not* an example URL.
### Are you using the latest version?
@ -42,11 +42,11 @@ Before reporting any issue, type `youtube-dl -U`. This should report that you're
### Is the issue already documented?
Make sure that someone has not already opened the issue you're trying to open. Search at the top of the window or browse the [GitHub Issues](https://github.com/rg3/youtube-dl/search?type=Issues) of this repository. If there is an issue, feel free to write something along the lines of "This affects me as well, with version 2015.01.01. Here is some more information on the issue: ...". While some issues may be old, a new post into them often spurs rapid activity.
Make sure that someone has not already opened the issue you're trying to open. Search at the top of the window or browse the [GitHub Issues](https://github.com/ytdl-org/youtube-dl/search?type=Issues) of this repository. If there is an issue, feel free to write something along the lines of "This affects me as well, with version 2015.01.01. Here is some more information on the issue: ...". While some issues may be old, a new post into them often spurs rapid activity.
### Why are existing options not enough?
Before requesting a new feature, please have a quick peek at [the list of supported options](https://github.com/rg3/youtube-dl/blob/master/README.md#options). Many feature requests are for features that actually exist already! Please, absolutely do show off your work in the issue report and detail how the existing similar options do *not* solve your problem.
Before requesting a new feature, please have a quick peek at [the list of supported options](https://github.com/ytdl-org/youtube-dl/blob/master/README.md#options). Many feature requests are for features that actually exist already! Please, absolutely do show off your work in the issue report and detail how the existing similar options do *not* solve your problem.
### Is there enough context in your bug report?
@ -70,7 +70,7 @@ It may sound strange, but some bug reports we receive are completely unrelated t
# DEVELOPER INSTRUCTIONS
Most users do not need to build youtube-dl and can [download the builds](http://rg3.github.io/youtube-dl/download.html) or get them from their distribution.
Most users do not need to build youtube-dl and can [download the builds](https://ytdl-org.github.io/youtube-dl/download.html) or get them from their distribution.
To run youtube-dl as a developer, you don't need to build anything either. Simply execute
@ -82,6 +82,8 @@ To run the test, simply invoke your favorite test runner, or execute a test file
python test/test_download.py
nosetests
See item 6 of [new extractor tutorial](#adding-support-for-a-new-site) for how to run extractor specific test cases.
If you want to create a build of youtube-dl yourself, you'll need
* python
@ -96,7 +98,7 @@ If you want to add support for a new site, first of all **make sure** this site
After you have ensured this site is distributing its content legally, you can follow this quick list (assuming your service is called `yourextractor`):
1. [Fork this repository](https://github.com/rg3/youtube-dl/fork)
1. [Fork this repository](https://github.com/ytdl-org/youtube-dl/fork)
2. Check out the source code with:
git clone git@github.com:YOUR_GITHUB_USERNAME/youtube-dl.git
@ -118,7 +120,7 @@ After you have ensured this site is distributing its content legally, you can fo
class YourExtractorIE(InfoExtractor):
_VALID_URL = r'https?://(?:www\.)?yourextractor\.com/watch/(?P<id>[0-9]+)'
_TEST = {
'url': 'http://yourextractor.com/watch/42',
'url': 'https://yourextractor.com/watch/42',
'md5': 'TODO: md5 sum of the first 10241 bytes of the video file (use --test)',
'info_dict': {
'id': '42',
@ -148,18 +150,22 @@ After you have ensured this site is distributing its content legally, you can fo
# TODO more properties (see youtube_dl/extractor/common.py)
}
```
5. Add an import in [`youtube_dl/extractor/extractors.py`](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/extractors.py).
6. Run `python test/test_download.py TestDownload.test_YourExtractor`. This *should fail* at first, but you can continually re-run it until you're done. If you decide to add more than one test, then rename ``_TEST`` to ``_TESTS`` and make it into a list of dictionaries. The tests will then be named `TestDownload.test_YourExtractor`, `TestDownload.test_YourExtractor_1`, `TestDownload.test_YourExtractor_2`, etc.
7. Have a look at [`youtube_dl/extractor/common.py`](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/common.py) for possible helper methods and a [detailed description of what your extractor should and may return](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/common.py#L74-L252). Add tests and code for as many as you want.
8. Make sure your code follows [youtube-dl coding conventions](#youtube-dl-coding-conventions) and check the code with [flake8](https://pypi.python.org/pypi/flake8). Also make sure your code works under all [Python](http://www.python.org/) versions claimed supported by youtube-dl, namely 2.6, 2.7, and 3.2+.
9. When the tests pass, [add](http://git-scm.com/docs/git-add) the new files and [commit](http://git-scm.com/docs/git-commit) them and [push](http://git-scm.com/docs/git-push) the result, like this:
5. Add an import in [`youtube_dl/extractor/extractors.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/extractor/extractors.py).
6. Run `python test/test_download.py TestDownload.test_YourExtractor`. This *should fail* at first, but you can continually re-run it until you're done. If you decide to add more than one test, then rename ``_TEST`` to ``_TESTS`` and make it into a list of dictionaries. The tests will then be named `TestDownload.test_YourExtractor`, `TestDownload.test_YourExtractor_1`, `TestDownload.test_YourExtractor_2`, etc. Note that tests with `only_matching` key in test's dict are not counted in.
7. Have a look at [`youtube_dl/extractor/common.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/extractor/common.py) for possible helper methods and a [detailed description of what your extractor should and may return](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303). Add tests and code for as many as you want.
8. Make sure your code follows [youtube-dl coding conventions](#youtube-dl-coding-conventions) and check the code with [flake8](https://flake8.pycqa.org/en/latest/index.html#quickstart):
$ flake8 youtube_dl/extractor/yourextractor.py
9. Make sure your code works under all [Python](https://www.python.org/) versions claimed supported by youtube-dl, namely 2.6, 2.7, and 3.2+.
10. When the tests pass, [add](https://git-scm.com/docs/git-add) the new files and [commit](https://git-scm.com/docs/git-commit) them and [push](https://git-scm.com/docs/git-push) the result, like this:
$ git add youtube_dl/extractor/extractors.py
$ git add youtube_dl/extractor/yourextractor.py
$ git commit -m '[yourextractor] Add new extractor'
$ git push origin yourextractor
10. Finally, [create a pull request](https://help.github.com/articles/creating-a-pull-request). We'll then review and merge it.
11. Finally, [create a pull request](https://help.github.com/articles/creating-a-pull-request). We'll then review and merge it.
In any case, thank you very much for your contributions!
@ -171,7 +177,7 @@ Extractors are very fragile by nature since they depend on the layout of the sou
### Mandatory and optional metafields
For extraction to work youtube-dl relies on metadata your extractor extracts and provides to youtube-dl expressed by an [information dictionary](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/common.py#L75-L257) or simply *info dict*. Only the following meta fields in the *info dict* are considered mandatory for a successful extraction process by youtube-dl:
For extraction to work youtube-dl relies on metadata your extractor extracts and provides to youtube-dl expressed by an [information dictionary](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303) or simply *info dict*. Only the following meta fields in the *info dict* are considered mandatory for a successful extraction process by youtube-dl:
- `id` (media identifier)
- `title` (media title)
@ -179,7 +185,7 @@ For extraction to work youtube-dl relies on metadata your extractor extracts and
In fact only the last option is technically mandatory (i.e. if you can't figure out the download location of the media the extraction does not make any sense). But by convention youtube-dl also treats `id` and `title` as mandatory. Thus the aforementioned metafields are the critical data that the extraction does not make any sense without and if any of them fail to be extracted then the extractor is considered completely broken.
[Any field](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/common.py#L149-L257) apart from the aforementioned ones are considered **optional**. That means that extraction should be **tolerant** to situations when sources for these fields can potentially be unavailable (even if they are always available at the moment) and **future-proof** in order not to break the extraction of general purpose mandatory fields.
[Any field](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L188-L303) apart from the aforementioned ones are considered **optional**. That means that extraction should be **tolerant** to situations when sources for these fields can potentially be unavailable (even if they are always available at the moment) and **future-proof** in order not to break the extraction of general purpose mandatory fields.
#### Example
@ -255,11 +261,33 @@ title = meta.get('title') or self._og_search_title(webpage)
This code will try to extract from `meta` first and if it fails it will try extracting `og:title` from a `webpage`.
### Make regular expressions flexible
### Regular expressions
When using regular expressions try to write them fuzzy and flexible.
#### Don't capture groups you don't use
#### Example
Capturing group must be an indication that it's used somewhere in the code. Any group that is not used must be non capturing.
##### Example
Don't capture id attribute name here since you can't use it for anything anyway.
Correct:
```python
r'(?:id|ID)=(?P<id>\d+)'
```
Incorrect:
```python
r'(id|ID)=(?P<id>\d+)'
```
#### Make regular expressions relaxed and flexible
When using regular expressions try to write them fuzzy, relaxed and flexible, skipping insignificant parts that are more likely to change, allowing both single and double quotes for quoted values and so on.
##### Example
Say you need to extract `title` from the following HTML code:
@ -292,7 +320,115 @@ title = self._search_regex(
webpage, 'title', group='title')
```
### Use safe conversion functions
### Long lines policy
Wrap all extracted numeric data into safe functions from `utils`: `int_or_none`, `float_or_none`. Use them for string to number conversions as well.
There is a soft limit to keep lines of code under 80 characters long. This means it should be respected if possible and if it does not make readability and code maintenance worse.
For example, you should **never** split long string literals like URLs or some other often copied entities over multiple lines to fit this limit:
Correct:
```python
'https://www.youtube.com/watch?v=FqZTN594JQw&list=PLMYEtVRpaqY00V9W81Cwmzp6N6vZqfUKD4'
```
Incorrect:
```python
'https://www.youtube.com/watch?v=FqZTN594JQw&list='
'PLMYEtVRpaqY00V9W81Cwmzp6N6vZqfUKD4'
```
### Inline values
Extracting variables is acceptable for reducing code duplication and improving readability of complex expressions. However, you should avoid extracting variables used only once and moving them to opposite parts of the extractor file, which makes reading the linear flow difficult.
#### Example
Correct:
```python
title = self._html_search_regex(r'<title>([^<]+)</title>', webpage, 'title')
```
Incorrect:
```python
TITLE_RE = r'<title>([^<]+)</title>'
# ...some lines of code...
title = self._html_search_regex(TITLE_RE, webpage, 'title')
```
### Collapse fallbacks
Multiple fallback values can quickly become unwieldy. Collapse multiple fallback values into a single expression via a list of patterns.
#### Example
Good:
```python
description = self._html_search_meta(
['og:description', 'description', 'twitter:description'],
webpage, 'description', default=None)
```
Unwieldy:
```python
description = (
self._og_search_description(webpage, default=None)
or self._html_search_meta('description', webpage, default=None)
or self._html_search_meta('twitter:description', webpage, default=None))
```
Methods supporting list of patterns are: `_search_regex`, `_html_search_regex`, `_og_search_property`, `_html_search_meta`.
### Trailing parentheses
Always move trailing parentheses after the last argument.
#### Example
Correct:
```python
lambda x: x['ResultSet']['Result'][0]['VideoUrlSet']['VideoUrl'],
list)
```
Incorrect:
```python
lambda x: x['ResultSet']['Result'][0]['VideoUrlSet']['VideoUrl'],
list,
)
```
### Use convenience conversion and parsing functions
Wrap all extracted numeric data into safe functions from [`youtube_dl/utils.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/utils.py): `int_or_none`, `float_or_none`. Use them for string to number conversions as well.
Use `url_or_none` for safe URL processing.
Use `try_get` for safe metadata extraction from parsed JSON.
Use `unified_strdate` for uniform `upload_date` or any `YYYYMMDD` meta field extraction, `unified_timestamp` for uniform `timestamp` extraction, `parse_filesize` for `filesize` extraction, `parse_count` for count meta fields extraction, `parse_resolution`, `parse_duration` for `duration` extraction, `parse_age_limit` for `age_limit` extraction.
Explore [`youtube_dl/utils.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/utils.py) for more useful convenience functions.
#### More examples
##### Safely extract optional description from parsed JSON
```python
description = try_get(response, lambda x: x['result']['video'][0]['summary'], compat_str)
```
##### Safely extract more optional metadata
```python
video = try_get(response, lambda x: x['result']['video'][0], dict) or {}
description = video.get('summary')
duration = float_or_none(video.get('durationMs'), scale=1000)
view_count = int_or_none(video.get('views'))
```

4034
ChangeLog

File diff suppressed because it is too large Load Diff

View File

@ -1,7 +1,9 @@
include README.md
include test/*.py
include test/*.json
include LICENSE
include AUTHORS
include ChangeLog
include youtube-dl.bash-completion
include youtube-dl.fish
include youtube-dl.1
recursive-include docs Makefile conf.py *.rst
recursive-include test *

View File

@ -1,7 +1,7 @@
all: youtube-dl README.md CONTRIBUTING.md README.txt youtube-dl.1 youtube-dl.bash-completion youtube-dl.zsh youtube-dl.fish supportedsites
clean:
rm -rf youtube-dl.1.temp.md youtube-dl.1 youtube-dl.bash-completion README.txt MANIFEST build/ dist/ .coverage cover/ youtube-dl.tar.gz youtube-dl.zsh youtube-dl.fish youtube_dl/extractor/lazy_extractors.py *.dump *.part* *.info.json *.mp4 *.m4a *.flv *.mp3 *.avi *.mkv *.webm *.3gp *.wav *.ape *.swf *.jpg *.png CONTRIBUTING.md.tmp ISSUE_TEMPLATE.md.tmp youtube-dl youtube-dl.exe
rm -rf youtube-dl.1.temp.md youtube-dl.1 youtube-dl.bash-completion README.txt MANIFEST build/ dist/ .coverage cover/ youtube-dl.tar.gz youtube-dl.zsh youtube-dl.fish youtube_dl/extractor/lazy_extractors.py *.dump *.part* *.ytdl *.info.json *.mp4 *.m4a *.flv *.mp3 *.avi *.mkv *.webm *.3gp *.wav *.ape *.swf *.jpg *.png CONTRIBUTING.md.tmp youtube-dl youtube-dl.exe
find . -name "*.pyc" -delete
find . -name "*.class" -delete
@ -14,6 +14,9 @@ PYTHON ?= /usr/bin/env python
# set SYSCONFDIR to /etc if PREFIX=/usr or PREFIX=/usr/local
SYSCONFDIR = $(shell if [ $(PREFIX) = /usr -o $(PREFIX) = /usr/local ]; then echo /etc; else echo $(PREFIX)/etc; fi)
# set markdown input format to "markdown-smart" for pandoc version 2 and to "markdown" for pandoc prior to version 2
MARKDOWN = $(shell if [ `pandoc -v | head -n1 | cut -d" " -f2 | head -c1` = "2" ]; then echo markdown-smart; else echo markdown; fi)
install: youtube-dl youtube-dl.1 youtube-dl.bash-completion youtube-dl.zsh youtube-dl.fish
install -d $(DESTDIR)$(BINDIR)
install -m 755 youtube-dl $(DESTDIR)$(BINDIR)
@ -36,8 +39,17 @@ test:
ot: offlinetest
# Keep this list in sync with devscripts/run_tests.sh
offlinetest: codetest
$(PYTHON) -m nose --verbose test --exclude test_download.py --exclude test_age_restriction.py --exclude test_subtitles.py --exclude test_write_annotations.py --exclude test_youtube_lists.py --exclude test_iqiyi_sdk_interpreter.py --exclude test_socks.py
$(PYTHON) -m nose --verbose test \
--exclude test_age_restriction.py \
--exclude test_download.py \
--exclude test_iqiyi_sdk_interpreter.py \
--exclude test_socks.py \
--exclude test_subtitles.py \
--exclude test_write_annotations.py \
--exclude test_youtube_lists.py \
--exclude test_youtube_signature.py
tar: youtube-dl.tar.gz
@ -46,8 +58,15 @@ tar: youtube-dl.tar.gz
pypi-files: youtube-dl.bash-completion README.txt youtube-dl.1 youtube-dl.fish
youtube-dl: youtube_dl/*.py youtube_dl/*/*.py
zip --quiet youtube-dl youtube_dl/*.py youtube_dl/*/*.py
zip --quiet --junk-paths youtube-dl youtube_dl/__main__.py
mkdir -p zip
for d in youtube_dl youtube_dl/downloader youtube_dl/extractor youtube_dl/postprocessor ; do \
mkdir -p zip/$$d ;\
cp -pPR $$d/*.py zip/$$d/ ;\
done
touch -t 200001010101 zip/youtube_dl/*.py zip/youtube_dl/*/*.py
mv zip/youtube_dl/__main__.py zip/
cd zip ; zip -q ../youtube-dl youtube_dl/*.py youtube_dl/*/*.py __main__.py
rm -rf zip
echo '#!$(PYTHON)' > youtube-dl
cat youtube-dl.zip >> youtube-dl
rm youtube-dl.zip
@ -59,18 +78,22 @@ README.md: youtube_dl/*.py youtube_dl/*/*.py
CONTRIBUTING.md: README.md
$(PYTHON) devscripts/make_contributing.py README.md CONTRIBUTING.md
.github/ISSUE_TEMPLATE.md: devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl.md youtube_dl/version.py
$(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl.md .github/ISSUE_TEMPLATE.md
issuetemplates: devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/1_broken_site.md .github/ISSUE_TEMPLATE_tmpl/2_site_support_request.md .github/ISSUE_TEMPLATE_tmpl/3_site_feature_request.md .github/ISSUE_TEMPLATE_tmpl/4_bug_report.md .github/ISSUE_TEMPLATE_tmpl/5_feature_request.md youtube_dl/version.py
$(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/1_broken_site.md .github/ISSUE_TEMPLATE/1_broken_site.md
$(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/2_site_support_request.md .github/ISSUE_TEMPLATE/2_site_support_request.md
$(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/3_site_feature_request.md .github/ISSUE_TEMPLATE/3_site_feature_request.md
$(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/4_bug_report.md .github/ISSUE_TEMPLATE/4_bug_report.md
$(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/5_feature_request.md .github/ISSUE_TEMPLATE/5_feature_request.md
supportedsites:
$(PYTHON) devscripts/make_supportedsites.py docs/supportedsites.md
README.txt: README.md
pandoc -f markdown -t plain README.md -o README.txt
pandoc -f $(MARKDOWN) -t plain README.md -o README.txt
youtube-dl.1: README.md
$(PYTHON) devscripts/prepare_manpage.py youtube-dl.1.temp.md
pandoc -s -f markdown -t man youtube-dl.1.temp.md -o youtube-dl.1
pandoc -s -f $(MARKDOWN) -t man youtube-dl.1.temp.md -o youtube-dl.1
rm -f youtube-dl.1.temp.md
youtube-dl.bash-completion: youtube_dl/*.py youtube_dl/*/*.py devscripts/bash-completion.in
@ -94,20 +117,19 @@ _EXTRACTOR_FILES = $(shell find youtube_dl/extractor -iname '*.py' -and -not -in
youtube_dl/extractor/lazy_extractors.py: devscripts/make_lazy_extractors.py devscripts/lazy_load_template.py $(_EXTRACTOR_FILES)
$(PYTHON) devscripts/make_lazy_extractors.py $@
youtube-dl.tar.gz: youtube-dl README.md README.txt youtube-dl.1 youtube-dl.bash-completion youtube-dl.zsh youtube-dl.fish ChangeLog
youtube-dl.tar.gz: youtube-dl README.md README.txt youtube-dl.1 youtube-dl.bash-completion youtube-dl.zsh youtube-dl.fish ChangeLog AUTHORS
@tar -czf youtube-dl.tar.gz --transform "s|^|youtube-dl/|" --owner 0 --group 0 \
--exclude '*.DS_Store' \
--exclude '*.kate-swp' \
--exclude '*.pyc' \
--exclude '*.pyo' \
--exclude '*~' \
--exclude '__pycache' \
--exclude '__pycache__' \
--exclude '.git' \
--exclude 'testdata' \
--exclude 'docs/_build' \
-- \
bin devscripts test youtube_dl docs \
ChangeLog LICENSE README.md README.txt \
ChangeLog AUTHORS LICENSE README.md README.txt \
Makefile MANIFEST.in youtube-dl.1 youtube-dl.bash-completion \
youtube-dl.zsh youtube-dl.fish setup.py \
youtube-dl.zsh youtube-dl.fish setup.py setup.cfg \
youtube-dl

532
README.md
View File

@ -1,3 +1,5 @@
[![Build Status](https://travis-ci.org/ytdl-org/youtube-dl.svg?branch=master)](https://travis-ci.org/ytdl-org/youtube-dl)
youtube-dl - download videos from youtube.com or other video platforms
- [INSTALLATION](#installation)
@ -15,7 +17,7 @@ youtube-dl - download videos from youtube.com or other video platforms
# INSTALLATION
To install it right away for all UNIX users (Linux, OS X, etc.), type:
To install it right away for all UNIX users (Linux, macOS, etc.), type:
sudo curl -L https://yt-dl.org/downloads/latest/youtube-dl -o /usr/local/bin/youtube-dl
sudo chmod a+rx /usr/local/bin/youtube-dl
@ -25,7 +27,7 @@ If you do not have curl, you can alternatively use a recent wget:
sudo wget https://yt-dl.org/downloads/latest/youtube-dl -O /usr/local/bin/youtube-dl
sudo chmod a+rx /usr/local/bin/youtube-dl
Windows users can [download an .exe file](https://yt-dl.org/latest/youtube-dl.exe) and place it in any location on their [PATH](http://en.wikipedia.org/wiki/PATH_%28variable%29) except for `%SYSTEMROOT%\System32` (e.g. **do not** put in `C:\Windows\System32`).
Windows users can [download an .exe file](https://yt-dl.org/latest/youtube-dl.exe) and place it in any location on their [PATH](https://en.wikipedia.org/wiki/PATH_%28variable%29) except for `%SYSTEMROOT%\System32` (e.g. **do not** put in `C:\Windows\System32`).
You can also use pip:
@ -33,7 +35,7 @@ You can also use pip:
This command will update youtube-dl if you have already installed it. See the [pypi page](https://pypi.python.org/pypi/youtube_dl) for more information.
OS X users can install youtube-dl with [Homebrew](http://brew.sh/):
macOS users can install youtube-dl with [Homebrew](https://brew.sh/):
brew install youtube-dl
@ -41,10 +43,10 @@ Or with [MacPorts](https://www.macports.org/):
sudo port install youtube-dl
Alternatively, refer to the [developer instructions](#developer-instructions) for how to check out and work with the git repository. For further options, including PGP signatures, see the [youtube-dl Download Page](https://rg3.github.io/youtube-dl/download.html).
Alternatively, refer to the [developer instructions](#developer-instructions) for how to check out and work with the git repository. For further options, including PGP signatures, see the [youtube-dl Download Page](https://ytdl-org.github.io/youtube-dl/download.html).
# DESCRIPTION
**youtube-dl** is a command-line program to download videos from YouTube.com and a few more sites. It requires the Python interpreter, version 2.6, 2.7, or 3.2+, and it is not platform specific. It should work on your Unix box, on Windows or on Mac OS X. It is released to the public domain, which means you can modify it, redistribute it or use it however you like.
**youtube-dl** is a command-line program to download videos from YouTube.com and a few more sites. It requires the Python interpreter, version 2.6, 2.7, or 3.2+, and it is not platform specific. It should work on your Unix box, on Windows or on macOS. It is released to the public domain, which means you can modify it, redistribute it or use it however you like.
youtube-dl [OPTIONS] URL [URL...]
@ -91,19 +93,31 @@ Alternatively, refer to the [developer instructions](#developer-instructions) fo
## Network Options:
--proxy URL Use the specified HTTP/HTTPS/SOCKS proxy.
To enable experimental SOCKS proxy, specify
a proper scheme. For example
To enable SOCKS proxy, specify a proper
scheme. For example
socks5://127.0.0.1:1080/. Pass in an empty
string (--proxy "") for direct connection
--socket-timeout SECONDS Time to wait before giving up, in seconds
--source-address IP Client-side IP address to bind to
-4, --force-ipv4 Make all connections via IPv4
-6, --force-ipv6 Make all connections via IPv6
## Geo Restriction:
--geo-verification-proxy URL Use this proxy to verify the IP address for
some geo-restricted sites. The default
proxy specified by --proxy (or none, if the
options is not present) is used for the
option is not present) is used for the
actual downloading.
--geo-bypass Bypass geographic restriction via faking
X-Forwarded-For HTTP header
--no-geo-bypass Do not bypass geographic restriction via
faking X-Forwarded-For HTTP header
--geo-bypass-country CODE Force bypass geographic restriction with
explicitly provided two-letter ISO 3166-2
country code
--geo-bypass-ip-block IP_BLOCK Force bypass geographic restriction with
explicitly provided IP block in CIDR
notation
## Video Selection:
--playlist-start NUMBER Playlist video to start at (default is 1)
@ -135,22 +149,24 @@ Alternatively, refer to the [developer instructions](#developer-instructions) fo
--max-views COUNT Do not download any videos with more than
COUNT views
--match-filter FILTER Generic video filter. Specify any key (see
help for -o for a list of available keys)
to match if the key is present, !key to
check if the key is not present,key >
NUMBER (like "comment_count > 12", also
works with >=, <, <=, !=, =) to compare
against a number, and & to require multiple
matches. Values which are not known are
excluded unless you put a question mark (?)
after the operator.For example, to only
match videos that have been liked more than
100 times and disliked less than 50 times
(or the dislike functionality is not
available at the given service), but who
also have a description, use --match-filter
"like_count > 100 & dislike_count <? 50 &
description" .
the "OUTPUT TEMPLATE" for a list of
available keys) to match if the key is
present, !key to check if the key is not
present, key > NUMBER (like "comment_count
> 12", also works with >=, <, <=, !=, =) to
compare against a number, key = 'LITERAL'
(like "uploader = 'Mike Smith'", also works
with !=) to match against a string literal
and & to require multiple matches. Values
which are not known are excluded unless you
put a question mark (?) after the operator.
For example, to only match videos that have
been liked more than 100 times and disliked
less than 50 times (or the dislike
functionality is not available at the given
service), but who also have a description,
use --match-filter "like_count > 100 &
dislike_count <? 50 & description" .
--no-playlist Download only the video, if the URL refers
to a video and a playlist.
--yes-playlist Download the playlist, if the URL refers to
@ -169,22 +185,30 @@ Alternatively, refer to the [developer instructions](#developer-instructions) fo
-R, --retries RETRIES Number of retries (default is 10), or
"infinite".
--fragment-retries RETRIES Number of retries for a fragment (default
is 10), or "infinite" (DASH and hlsnative
only)
--skip-unavailable-fragments Skip unavailable fragments (DASH and
hlsnative only)
is 10), or "infinite" (DASH, hlsnative and
ISM)
--skip-unavailable-fragments Skip unavailable fragments (DASH, hlsnative
and ISM)
--abort-on-unavailable-fragment Abort downloading when some fragment is not
available
--keep-fragments Keep downloaded fragments on disk after
downloading is finished; fragments are
erased by default
--buffer-size SIZE Size of download buffer (e.g. 1024 or 16K)
(default is 1024)
--no-resize-buffer Do not automatically adjust the buffer
size. By default, the buffer size is
automatically resized from an initial value
of SIZE.
--http-chunk-size SIZE Size of a chunk for chunk-based HTTP
downloading (e.g. 10485760 or 10M) (default
is disabled). May be useful for bypassing
bandwidth throttling imposed by a webserver
(experimental)
--playlist-reverse Download playlist videos in reverse order
--playlist-random Download playlist videos in random order
--xattr-set-filesize Set file xattribute ytdl.filesize with
expected file size (experimental)
expected file size
--hls-prefer-native Use the native HLS downloader instead of
ffmpeg
--hls-prefer-ffmpeg Use ffmpeg instead of the native HLS
@ -201,25 +225,17 @@ Alternatively, refer to the [developer instructions](#developer-instructions) fo
## Filesystem Options:
-a, --batch-file FILE File containing URLs to download ('-' for
stdin)
stdin), one URL per line. Lines starting
with '#', ';' or ']' are considered as
comments and ignored.
--id Use only video ID in file name
-o, --output TEMPLATE Output filename template, see the "OUTPUT
TEMPLATE" for all the info
--autonumber-size NUMBER Specify the number of digits in
%(autonumber)s when it is present in output
filename template or --auto-number option
is given (default is 5)
--autonumber-start NUMBER Specify the start value for %(autonumber)s
(default is 1)
--restrict-filenames Restrict filenames to only ASCII
characters, and avoid "&" and spaces in
filenames
-A, --auto-number [deprecated; use -o
"%(autonumber)s-%(title)s.%(ext)s" ] Number
downloaded files starting from 00000
-t, --title [deprecated] Use title in file name
(default)
-l, --literal [deprecated] Alias of --title
-w, --no-overwrites Do not overwrite files
-c, --continue Force resume of partially downloaded files.
By default, youtube-dl will resume
@ -272,8 +288,8 @@ Alternatively, refer to the [developer instructions](#developer-instructions) fo
--get-filename Simulate, quiet but print output filename
--get-format Simulate, quiet but print output format
-j, --dump-json Simulate, quiet but print JSON information.
See --output for a description of available
keys.
See the "OUTPUT TEMPLATE" for a description
of available keys.
-J, --dump-single-json Simulate, quiet but print JSON information
for each command-line argument. If the URL
refers to a playlist, dump the whole
@ -373,8 +389,9 @@ Alternatively, refer to the [developer instructions](#developer-instructions) fo
(requires ffmpeg or avconv and ffprobe or
avprobe)
--audio-format FORMAT Specify audio format: "best", "aac",
"vorbis", "mp3", "m4a", "opus", or "wav";
"best" by default; No effect without -x
"flac", "mp3", "m4a", "opus", "vorbis", or
"wav"; "best" by default; No effect without
-x
--audio-quality QUALITY Specify ffmpeg/avconv audio quality, insert
a value between 0 (better) and 9 (worse)
for VBR or a specific bitrate like 128K
@ -394,12 +411,14 @@ Alternatively, refer to the [developer instructions](#developer-instructions) fo
--add-metadata Write metadata to the video file
--metadata-from-title FORMAT Parse additional metadata like song title /
artist from the video title. The format
syntax is the same as --output, the parsed
parameters replace existing values.
Additional templates: %(album)s,
%(artist)s. Example: --metadata-from-title
"%(artist)s - %(title)s" matches a title
like "Coldplay - Paradise"
syntax is the same as --output. Regular
expression with named capture groups may
also be used. The parsed parameters replace
existing values. Example: --metadata-from-
title "%(artist)s - %(title)s" matches a
title like "Coldplay - Paradise". Example
(regex): --metadata-from-title
"(?P<artist>.+?) - (?P<title>.+)"
--xattrs Write metadata to the video file's xattrs
(using dublin core and xdg standards)
--fixup POLICY Automatically correct known faults of the
@ -408,22 +427,22 @@ Alternatively, refer to the [developer instructions](#developer-instructions) fo
default; fix file if we can, warn
otherwise)
--prefer-avconv Prefer avconv over ffmpeg for running the
postprocessors (default)
--prefer-ffmpeg Prefer ffmpeg over avconv for running the
postprocessors
--prefer-ffmpeg Prefer ffmpeg over avconv for running the
postprocessors (default)
--ffmpeg-location PATH Location of the ffmpeg/avconv binary;
either the path to the binary or its
containing directory.
--exec CMD Execute a command on the file after
downloading, similar to find's -exec
syntax. Example: --exec 'adb push {}
/sdcard/Music/ && rm {}'
downloading and post-processing, similar to
find's -exec syntax. Example: --exec 'adb
push {} /sdcard/Music/ && rm {}'
--convert-subs FORMAT Convert the subtitles to other format
(currently supported: srt|ass|vtt)
(currently supported: srt|ass|vtt|lrc)
# CONFIGURATION
You can configure youtube-dl by placing any supported command line option to a configuration file. On Linux and OS X, the system wide configuration file is located at `/etc/youtube-dl.conf` and the user wide configuration file at `~/.config/youtube-dl/config`. On Windows, the user wide configuration file locations are `%APPDATA%\youtube-dl\config.txt` or `C:\Users\<user name>\youtube-dl.conf`. Note that by default configuration file may not exist so you may need to create it yourself.
You can configure youtube-dl by placing any supported command line option to a configuration file. On Linux and macOS, the system wide configuration file is located at `/etc/youtube-dl.conf` and the user wide configuration file at `~/.config/youtube-dl/config`. On Windows, the user wide configuration file locations are `%APPDATA%\youtube-dl\config.txt` or `C:\Users\<user name>\youtube-dl.conf`. Note that by default configuration file may not exist so you may need to create it yourself.
For example, with the following configuration file youtube-dl will always extract the audio, not copy the mtime, use a proxy and save all videos under `Movies` directory in your home directory:
```
@ -450,7 +469,7 @@ You can also use `--config-location` if you want to use custom configuration fil
### Authentication with `.netrc` file
You may also want to configure automatic credentials storage for extractors that support authentication (by providing login and password with `--username` and `--password`) in order not to pass credentials as command line arguments on every youtube-dl execution and prevent tracking plain text passwords in the shell command history. You can achieve this using a [`.netrc` file](http://stackoverflow.com/tags/.netrc/info) on a per extractor basis. For that you will need to create a `.netrc` file in your `$HOME` and restrict permissions to read/write by only you:
You may also want to configure automatic credentials storage for extractors that support authentication (by providing login and password with `--username` and `--password`) in order not to pass credentials as command line arguments on every youtube-dl execution and prevent tracking plain text passwords in the shell command history. You can achieve this using a [`.netrc` file](https://stackoverflow.com/tags/.netrc/info) on a per extractor basis. For that you will need to create a `.netrc` file in your `$HOME` and restrict permissions to read/write by only you:
```
touch $HOME/.netrc
chmod a-rwx,u+rw $HOME/.netrc
@ -466,7 +485,10 @@ machine twitch login my_twitch_account_name password my_twitch_password
```
To activate authentication with the `.netrc` file you should pass `--netrc` to youtube-dl or place it in the [configuration file](#configuration).
On Windows you may also need to setup the `%HOME%` environment variable manually.
On Windows you may also need to setup the `%HOME%` environment variable manually. For example:
```
set HOME=%USERPROFILE%
```
# OUTPUT TEMPLATE
@ -474,87 +496,98 @@ The `-o` option allows users to indicate a template for the output file names.
**tl;dr:** [navigate me to examples](#output-template-examples).
The basic usage is not to set any template arguments when downloading a single file, like in `youtube-dl -o funny_video.flv "http://some/video"`. However, it may contain special sequences that will be replaced when downloading each video. The special sequences have the format `%(NAME)s`. To clarify, that is a percent symbol followed by a name in parentheses, followed by a lowercase S. Allowed names are:
- `id`: Video identifier
- `title`: Video title
- `url`: Video URL
- `ext`: Video filename extension
- `alt_title`: A secondary title of the video
- `display_id`: An alternative identifier for the video
- `uploader`: Full name of the video uploader
- `license`: License name the video is licensed under
- `creator`: The creator of the video
- `release_date`: The date (YYYYMMDD) when the video was released
- `timestamp`: UNIX timestamp of the moment the video became available
- `upload_date`: Video upload date (YYYYMMDD)
- `uploader_id`: Nickname or id of the video uploader
- `location`: Physical location where the video was filmed
- `duration`: Length of the video in seconds
- `view_count`: How many users have watched the video on the platform
- `like_count`: Number of positive ratings of the video
- `dislike_count`: Number of negative ratings of the video
- `repost_count`: Number of reposts of the video
- `average_rating`: Average rating give by users, the scale used depends on the webpage
- `comment_count`: Number of comments on the video
- `age_limit`: Age restriction for the video (years)
- `format`: A human-readable description of the format
- `format_id`: Format code specified by `--format`
- `format_note`: Additional info about the format
- `width`: Width of the video
- `height`: Height of the video
- `resolution`: Textual description of width and height
- `tbr`: Average bitrate of audio and video in KBit/s
- `abr`: Average audio bitrate in KBit/s
- `acodec`: Name of the audio codec in use
- `asr`: Audio sampling rate in Hertz
- `vbr`: Average video bitrate in KBit/s
- `fps`: Frame rate
- `vcodec`: Name of the video codec in use
- `container`: Name of the container format
- `filesize`: The number of bytes, if known in advance
- `filesize_approx`: An estimate for the number of bytes
- `protocol`: The protocol that will be used for the actual download
- `extractor`: Name of the extractor
- `extractor_key`: Key name of the extractor
- `epoch`: Unix epoch when creating the file
- `autonumber`: Five-digit number that will be increased with each download, starting at zero
- `playlist`: Name or id of the playlist that contains the video
- `playlist_index`: Index of the video in the playlist padded with leading zeros according to the total length of the playlist
- `playlist_id`: Playlist identifier
- `playlist_title`: Playlist title
The basic usage is not to set any template arguments when downloading a single file, like in `youtube-dl -o funny_video.flv "https://some/video"`. However, it may contain special sequences that will be replaced when downloading each video. The special sequences may be formatted according to [python string formatting operations](https://docs.python.org/2/library/stdtypes.html#string-formatting). For example, `%(NAME)s` or `%(NAME)05d`. To clarify, that is a percent symbol followed by a name in parentheses, followed by formatting operations. Allowed names along with sequence type are:
- `id` (string): Video identifier
- `title` (string): Video title
- `url` (string): Video URL
- `ext` (string): Video filename extension
- `alt_title` (string): A secondary title of the video
- `display_id` (string): An alternative identifier for the video
- `uploader` (string): Full name of the video uploader
- `license` (string): License name the video is licensed under
- `creator` (string): The creator of the video
- `release_date` (string): The date (YYYYMMDD) when the video was released
- `timestamp` (numeric): UNIX timestamp of the moment the video became available
- `upload_date` (string): Video upload date (YYYYMMDD)
- `uploader_id` (string): Nickname or id of the video uploader
- `channel` (string): Full name of the channel the video is uploaded on
- `channel_id` (string): Id of the channel
- `location` (string): Physical location where the video was filmed
- `duration` (numeric): Length of the video in seconds
- `view_count` (numeric): How many users have watched the video on the platform
- `like_count` (numeric): Number of positive ratings of the video
- `dislike_count` (numeric): Number of negative ratings of the video
- `repost_count` (numeric): Number of reposts of the video
- `average_rating` (numeric): Average rating give by users, the scale used depends on the webpage
- `comment_count` (numeric): Number of comments on the video
- `age_limit` (numeric): Age restriction for the video (years)
- `is_live` (boolean): Whether this video is a live stream or a fixed-length video
- `start_time` (numeric): Time in seconds where the reproduction should start, as specified in the URL
- `end_time` (numeric): Time in seconds where the reproduction should end, as specified in the URL
- `format` (string): A human-readable description of the format
- `format_id` (string): Format code specified by `--format`
- `format_note` (string): Additional info about the format
- `width` (numeric): Width of the video
- `height` (numeric): Height of the video
- `resolution` (string): Textual description of width and height
- `tbr` (numeric): Average bitrate of audio and video in KBit/s
- `abr` (numeric): Average audio bitrate in KBit/s
- `acodec` (string): Name of the audio codec in use
- `asr` (numeric): Audio sampling rate in Hertz
- `vbr` (numeric): Average video bitrate in KBit/s
- `fps` (numeric): Frame rate
- `vcodec` (string): Name of the video codec in use
- `container` (string): Name of the container format
- `filesize` (numeric): The number of bytes, if known in advance
- `filesize_approx` (numeric): An estimate for the number of bytes
- `protocol` (string): The protocol that will be used for the actual download
- `extractor` (string): Name of the extractor
- `extractor_key` (string): Key name of the extractor
- `epoch` (numeric): Unix epoch when creating the file
- `autonumber` (numeric): Number that will be increased with each download, starting at `--autonumber-start`
- `playlist` (string): Name or id of the playlist that contains the video
- `playlist_index` (numeric): Index of the video in the playlist padded with leading zeros according to the total length of the playlist
- `playlist_id` (string): Playlist identifier
- `playlist_title` (string): Playlist title
- `playlist_uploader` (string): Full name of the playlist uploader
- `playlist_uploader_id` (string): Nickname or id of the playlist uploader
Available for the video that belongs to some logical chapter or section:
- `chapter`: Name or title of the chapter the video belongs to
- `chapter_number`: Number of the chapter the video belongs to
- `chapter_id`: Id of the chapter the video belongs to
- `chapter` (string): Name or title of the chapter the video belongs to
- `chapter_number` (numeric): Number of the chapter the video belongs to
- `chapter_id` (string): Id of the chapter the video belongs to
Available for the video that is an episode of some series or programme:
- `series`: Title of the series or programme the video episode belongs to
- `season`: Title of the season the video episode belongs to
- `season_number`: Number of the season the video episode belongs to
- `season_id`: Id of the season the video episode belongs to
- `episode`: Title of the video episode
- `episode_number`: Number of the video episode within a season
- `episode_id`: Id of the video episode
- `series` (string): Title of the series or programme the video episode belongs to
- `season` (string): Title of the season the video episode belongs to
- `season_number` (numeric): Number of the season the video episode belongs to
- `season_id` (string): Id of the season the video episode belongs to
- `episode` (string): Title of the video episode
- `episode_number` (numeric): Number of the video episode within a season
- `episode_id` (string): Id of the video episode
Available for the media that is a track or a part of a music album:
- `track`: Title of the track
- `track_number`: Number of the track within an album or a disc
- `track_id`: Id of the track
- `artist`: Artist(s) of the track
- `genre`: Genre(s) of the track
- `album`: Title of the album the track belongs to
- `album_type`: Type of the album
- `album_artist`: List of all artists appeared on the album
- `disc_number`: Number of the disc or other physical medium the track belongs to
- `release_year`: Year (YYYY) when the album was released
- `track` (string): Title of the track
- `track_number` (numeric): Number of the track within an album or a disc
- `track_id` (string): Id of the track
- `artist` (string): Artist(s) of the track
- `genre` (string): Genre(s) of the track
- `album` (string): Title of the album the track belongs to
- `album_type` (string): Type of the album
- `album_artist` (string): List of all artists appeared on the album
- `disc_number` (numeric): Number of the disc or other physical medium the track belongs to
- `release_year` (numeric): Year (YYYY) when the album was released
Each aforementioned sequence when referenced in an output template will be replaced by the actual value corresponding to the sequence name. Note that some of the sequences are not guaranteed to be present since they depend on the metadata obtained by a particular extractor. Such sequences will be replaced with `NA`.
For example for `-o %(title)s-%(id)s.%(ext)s` and an mp4 video with title `youtube-dl test video` and id `BaW_jenozKcj`, this will result in a `youtube-dl test video-BaW_jenozKcj.mp4` file created in the current directory.
For numeric sequences you can use numeric related formatting, for example, `%(view_count)05d` will result in a string with view count padded with zeros up to 5 characters, like in `00042`.
Output templates can also contain arbitrary hierarchical path, e.g. `-o '%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s'` which will result in downloading each video in a directory corresponding to this path template. Any missing directory will be automatically created for you.
To use percent literals in an output template use `%%`. To output to stdout use `-o -`.
@ -569,7 +602,7 @@ If you are using an output template inside a Windows batch file then you must es
#### Output template examples
Note on Windows you may need to use double quotes instead of single.
Note that on Windows you may need to use double quotes instead of single.
```bash
$ youtube-dl --get-filename -o '%(title)s.%(ext)s' BaW_jenozKc
@ -588,7 +621,7 @@ $ youtube-dl -o '%(uploader)s/%(playlist)s/%(playlist_index)s - %(title)s.%(ext)
$ youtube-dl -u user -p password -o '~/MyVideos/%(playlist)s/%(chapter_number)s - %(chapter)s/%(title)s.%(ext)s' https://www.udemy.com/java-tutorial/
# Download entire series season keeping each series and each season in separate directory under C:/MyVideos
$ youtube-dl -o "C:/MyVideos/%(series)s/%(season_number)s - %(season)s/%(episode_number)s - %(episode)s.%(ext)s" http://videomore.ru/kino_v_detalayah/5_sezon/367617
$ youtube-dl -o "C:/MyVideos/%(series)s/%(season_number)s - %(season)s/%(episode_number)s - %(episode)s.%(ext)s" https://videomore.ru/kino_v_detalayah/5_sezon/367617
# Stream the video being downloaded to stdout
$ youtube-dl -o - BaW_jenozKc
@ -609,6 +642,7 @@ The simplest case is requesting a specific format, for example with `-f 22` you
You can also use a file extension (currently `3gp`, `aac`, `flv`, `m4a`, `mp3`, `mp4`, `ogg`, `wav`, `webm` are supported) to download the best quality format of a particular file extension served as a single file, e.g. `-f webm` will download the best quality format with the `webm` extension served as a single file.
You can also use special names to select particular edge case formats:
- `best`: Select the best quality format represented by a single file with video and audio.
- `worst`: Select the worst quality format represented by a single file with video and audio.
- `bestvideo`: Select the best quality video-only format (e.g. DASH video). May not be available.
@ -625,6 +659,7 @@ If you want to download several formats of the same video use a comma as a separ
You can also filter the video formats by putting a condition in brackets, as in `-f "best[height=720]"` (or `-f "[filesize>10M]"`).
The following numeric meta fields can be used with comparisons `<`, `<=`, `>`, `>=`, `=` (equals), `!=` (not equals):
- `filesize`: The number of bytes, if known in advance
- `width`: Width of the video, if known
- `height`: Height of the video, if known
@ -634,14 +669,17 @@ The following numeric meta fields can be used with comparisons `<`, `<=`, `>`, `
- `asr`: Audio sampling rate in Hertz
- `fps`: Frame rate
Also filtering work for comparisons `=` (equals), `!=` (not equals), `^=` (begins with), `$=` (ends with), `*=` (contains) and following string meta fields:
Also filtering work for comparisons `=` (equals), `^=` (starts with), `$=` (ends with), `*=` (contains) and following string meta fields:
- `ext`: File extension
- `acodec`: Name of the audio codec in use
- `vcodec`: Name of the video codec in use
- `container`: Name of the container format
- `protocol`: The protocol that will be used for the actual download, lower-case (`http`, `https`, `rtsp`, `rtmp`, `rtmpe`, `mms`, `f4m`, `ism`, `m3u8`, or `m3u8_native`)
- `protocol`: The protocol that will be used for the actual download, lower-case (`http`, `https`, `rtsp`, `rtmp`, `rtmpe`, `mms`, `f4m`, `ism`, `http_dash_segments`, `m3u8`, or `m3u8_native`)
- `format_id`: A short description of the format
Any string comparison may be prefixed with negation `!` in order to produce an opposite comparison, e.g. `!*=` (does not contain).
Note that none of the aforementioned meta fields are guaranteed to be present since this solely depends on the metadata obtained by particular extractor, i.e. the metadata offered by the video hoster.
Formats for which the value is not known are excluded unless you put a question mark (`?`) after the operator. You can combine format filters, so `-f "[height <=? 720][tbr>500]"` selects up to 720p videos (or videos where the height is not known) with a bitrate of at least 500 KBit/s.
@ -650,19 +688,19 @@ You can merge the video and audio of two formats into a single file using `-f <v
Format selectors can also be grouped using parentheses, for example if you want to download the best mp4 and webm formats with a height lower than 480 you can use `-f '(mp4,webm)[height<480]'`.
Since the end of April 2015 and version 2015.04.26, youtube-dl uses `-f bestvideo+bestaudio/best` as the default format selection (see [#5447](https://github.com/rg3/youtube-dl/issues/5447), [#5456](https://github.com/rg3/youtube-dl/issues/5456)). If ffmpeg or avconv are installed this results in downloading `bestvideo` and `bestaudio` separately and muxing them together into a single file giving the best overall quality available. Otherwise it falls back to `best` and results in downloading the best available quality served as a single file. `best` is also needed for videos that don't come from YouTube because they don't provide the audio and video in two different files. If you want to only download some DASH formats (for example if you are not interested in getting videos with a resolution higher than 1080p), you can add `-f bestvideo[height<=?1080]+bestaudio/best` to your configuration file. Note that if you use youtube-dl to stream to `stdout` (and most likely to pipe it to your media player then), i.e. you explicitly specify output template as `-o -`, youtube-dl still uses `-f best` format selection in order to start content delivery immediately to your player and not to wait until `bestvideo` and `bestaudio` are downloaded and muxed.
Since the end of April 2015 and version 2015.04.26, youtube-dl uses `-f bestvideo+bestaudio/best` as the default format selection (see [#5447](https://github.com/ytdl-org/youtube-dl/issues/5447), [#5456](https://github.com/ytdl-org/youtube-dl/issues/5456)). If ffmpeg or avconv are installed this results in downloading `bestvideo` and `bestaudio` separately and muxing them together into a single file giving the best overall quality available. Otherwise it falls back to `best` and results in downloading the best available quality served as a single file. `best` is also needed for videos that don't come from YouTube because they don't provide the audio and video in two different files. If you want to only download some DASH formats (for example if you are not interested in getting videos with a resolution higher than 1080p), you can add `-f bestvideo[height<=?1080]+bestaudio/best` to your configuration file. Note that if you use youtube-dl to stream to `stdout` (and most likely to pipe it to your media player then), i.e. you explicitly specify output template as `-o -`, youtube-dl still uses `-f best` format selection in order to start content delivery immediately to your player and not to wait until `bestvideo` and `bestaudio` are downloaded and muxed.
If you want to preserve the old format selection behavior (prior to youtube-dl 2015.04.26), i.e. you want to download the best available quality media served as a single file, you should explicitly specify your choice with `-f best`. You may want to add it to the [configuration file](#configuration) in order not to type it every time you run youtube-dl.
#### Format selection examples
Note on Windows you may need to use double quotes instead of single.
Note that on Windows you may need to use double quotes instead of single.
```bash
# Download best mp4 format available or any other best if no mp4 available
$ youtube-dl -f 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best'
# Download best format available but not better that 480p
# Download best format available but no better than 480p
$ youtube-dl -f 'bestvideo[height<=480]+bestaudio/best[height<=480]'
# Download best video only format but no bigger than 50 MB
@ -701,21 +739,21 @@ $ youtube-dl --dateafter 20000101 --datebefore 20091231
### How do I update youtube-dl?
If you've followed [our manual installation instructions](http://rg3.github.io/youtube-dl/download.html), you can simply run `youtube-dl -U` (or, on Linux, `sudo youtube-dl -U`).
If you've followed [our manual installation instructions](https://ytdl-org.github.io/youtube-dl/download.html), you can simply run `youtube-dl -U` (or, on Linux, `sudo youtube-dl -U`).
If you have used pip, a simple `sudo pip install -U youtube-dl` is sufficient to update.
If you have installed youtube-dl using a package manager like *apt-get* or *yum*, use the standard system update mechanism to update. Note that distribution packages are often outdated. As a rule of thumb, youtube-dl releases at least once a month, and often weekly or even daily. Simply go to http://yt-dl.org/ to find out the current version. Unfortunately, there is nothing we youtube-dl developers can do if your distribution serves a really outdated version. You can (and should) complain to your distribution in their bugtracker or support forum.
If you have installed youtube-dl using a package manager like *apt-get* or *yum*, use the standard system update mechanism to update. Note that distribution packages are often outdated. As a rule of thumb, youtube-dl releases at least once a month, and often weekly or even daily. Simply go to https://yt-dl.org to find out the current version. Unfortunately, there is nothing we youtube-dl developers can do if your distribution serves a really outdated version. You can (and should) complain to your distribution in their bugtracker or support forum.
As a last resort, you can also uninstall the version installed by your package manager and follow our manual installation instructions. For that, remove the distribution's package, with a line like
sudo apt-get remove -y youtube-dl
Afterwards, simply follow [our manual installation instructions](http://rg3.github.io/youtube-dl/download.html):
Afterwards, simply follow [our manual installation instructions](https://ytdl-org.github.io/youtube-dl/download.html):
```
sudo wget https://yt-dl.org/latest/youtube-dl -O /usr/local/bin/youtube-dl
sudo chmod a+x /usr/local/bin/youtube-dl
sudo wget https://yt-dl.org/downloads/latest/youtube-dl -O /usr/local/bin/youtube-dl
sudo chmod a+rx /usr/local/bin/youtube-dl
hash -r
```
@ -745,17 +783,17 @@ Most people asking this question are not aware that youtube-dl now defaults to d
### I get HTTP error 402 when trying to download a video. What's this?
Apparently YouTube requires you to pass a CAPTCHA test if you download too much. We're [considering to provide a way to let you solve the CAPTCHA](https://github.com/rg3/youtube-dl/issues/154), but at the moment, your best course of action is pointing a web browser to the youtube URL, solving the CAPTCHA, and restart youtube-dl.
Apparently YouTube requires you to pass a CAPTCHA test if you download too much. We're [considering to provide a way to let you solve the CAPTCHA](https://github.com/ytdl-org/youtube-dl/issues/154), but at the moment, your best course of action is pointing a web browser to the youtube URL, solving the CAPTCHA, and restart youtube-dl.
### Do I need any other programs?
youtube-dl works fine on its own on most sites. However, if you want to convert video/audio, you'll need [avconv](https://libav.org/) or [ffmpeg](https://www.ffmpeg.org/). On some sites - most notably YouTube - videos can be retrieved in a higher quality format without sound. youtube-dl will detect whether avconv/ffmpeg is present and automatically pick the best option.
Videos or video formats streamed via RTMP protocol can only be downloaded when [rtmpdump](https://rtmpdump.mplayerhq.hu/) is installed. Downloading MMS and RTSP videos requires either [mplayer](http://mplayerhq.hu/) or [mpv](https://mpv.io/) to be installed.
Videos or video formats streamed via RTMP protocol can only be downloaded when [rtmpdump](https://rtmpdump.mplayerhq.hu/) is installed. Downloading MMS and RTSP videos requires either [mplayer](https://mplayerhq.hu/) or [mpv](https://mpv.io/) to be installed.
### I have downloaded a video but how can I play it?
Once the video is fully downloaded, use any video player, such as [mpv](https://mpv.io/), [vlc](http://www.videolan.org/) or [mplayer](http://www.mplayerhq.hu/).
Once the video is fully downloaded, use any video player, such as [mpv](https://mpv.io/), [vlc](https://www.videolan.org/) or [mplayer](https://www.mplayerhq.hu/).
### I extracted a video URL with `-g`, but it does not play on another machine / in my web browser.
@ -797,7 +835,9 @@ In February 2015, the new YouTube player contained a character sequence in a str
### HTTP Error 429: Too Many Requests or 402: Payment Required
These two error codes indicate that the service is blocking your IP address because of overuse. Contact the service and ask them to unblock your IP address, or - if you have acquired a whitelisted IP address already - use the [`--proxy` or `--source-address` options](#network-options) to select another IP address.
These two error codes indicate that the service is blocking your IP address because of overuse. Usually this is a soft block meaning that you can gain access again after solving CAPTCHA. Just open a browser and solve a CAPTCHA the service suggests you and after that [pass cookies](#how-do-i-pass-cookies-to-youtube-dl) to youtube-dl. Note that if your machine has multiple external IPs then you should also pass exactly the same IP you've used for solving CAPTCHA with [`--source-address`](#network-options). Also you may need to pass a `User-Agent` HTTP header of your browser with [`--user-agent`](#workarounds).
If this is not the case (no CAPTCHA suggested to solve by the service) then you can contact the service and ask them to unblock your IP address, or - if you have acquired a whitelisted IP address already - use the [`--proxy` or `--source-address` options](#network-options) to select another IP address.
### SyntaxError: Non-ASCII character
@ -810,7 +850,7 @@ means you're using an outdated version of Python. Please update to Python 2.6 or
### What is this binary file? Where has the code gone?
Since June 2012 ([#342](https://github.com/rg3/youtube-dl/issues/342)) youtube-dl is packed as an executable zipfile, simply unzip it (might need renaming to `youtube-dl.zip` first on some systems) or clone the git repository, as laid out above. If you modify the code, you can run it by executing the `__main__.py` file. To recompile the executable, run `make youtube-dl`.
Since June 2012 ([#342](https://github.com/ytdl-org/youtube-dl/issues/342)) youtube-dl is packed as an executable zipfile, simply unzip it (might need renaming to `youtube-dl.zip` first on some systems) or clone the git repository, as laid out above. If you modify the code, you can run it by executing the `__main__.py` file. To recompile the executable, run `make youtube-dl`.
### The exe throws an error due to missing `MSVCR100.dll`
@ -830,26 +870,26 @@ Use the `-o` to specify an [output template](#output-template), for example `-o
### How do I download a video starting with a `-`?
Either prepend `http://www.youtube.com/watch?v=` or separate the ID from the options with `--`:
Either prepend `https://www.youtube.com/watch?v=` or separate the ID from the options with `--`:
youtube-dl -- -wNyEUrxzFU
youtube-dl "http://www.youtube.com/watch?v=-wNyEUrxzFU"
youtube-dl "https://www.youtube.com/watch?v=-wNyEUrxzFU"
### How do I pass cookies to youtube-dl?
Use the `--cookies` option, for example `--cookies /path/to/cookies/file.txt`.
In order to extract cookies from browser use any conforming browser extension for exporting cookies. For example, [cookies.txt](https://chrome.google.com/webstore/detail/cookiestxt/njabckikapfpffapmjgojcnbfjonfjfg) (for Chrome) or [Export Cookies](https://addons.mozilla.org/en-US/firefox/addon/export-cookies/) (for Firefox).
In order to extract cookies from browser use any conforming browser extension for exporting cookies. For example, [cookies.txt](https://chrome.google.com/webstore/detail/cookiestxt/njabckikapfpffapmjgojcnbfjonfjfg) (for Chrome) or [cookies.txt](https://addons.mozilla.org/en-US/firefox/addon/cookies-txt/) (for Firefox).
Note that the cookies file must be in Mozilla/Netscape format and the first line of the cookies file must be either `# HTTP Cookie File` or `# Netscape HTTP Cookie File`. Make sure you have correct [newline format](https://en.wikipedia.org/wiki/Newline) in the cookies file and convert newlines if necessary to correspond with your OS, namely `CRLF` (`\r\n`) for Windows and `LF` (`\n`) for Unix and Unix-like systems (Linux, Mac OS, etc.). `HTTP Error 400: Bad Request` when using `--cookies` is a good sign of invalid newline format.
Note that the cookies file must be in Mozilla/Netscape format and the first line of the cookies file must be either `# HTTP Cookie File` or `# Netscape HTTP Cookie File`. Make sure you have correct [newline format](https://en.wikipedia.org/wiki/Newline) in the cookies file and convert newlines if necessary to correspond with your OS, namely `CRLF` (`\r\n`) for Windows and `LF` (`\n`) for Unix and Unix-like systems (Linux, macOS, etc.). `HTTP Error 400: Bad Request` when using `--cookies` is a good sign of invalid newline format.
Passing cookies to youtube-dl is a good way to workaround login when a particular extractor does not implement it explicitly. Another use case is working around [CAPTCHA](https://en.wikipedia.org/wiki/CAPTCHA) some websites require you to solve in particular cases in order to get access (e.g. YouTube, CloudFlare).
### How do I stream directly to media player?
You will first need to tell youtube-dl to stream media to stdout with `-o -`, and also tell your media player to read from stdin (it must be capable of this for streaming) and then pipe former to latter. For example, streaming to [vlc](http://www.videolan.org/) can be achieved with:
You will first need to tell youtube-dl to stream media to stdout with `-o -`, and also tell your media player to read from stdin (it must be capable of this for streaming) and then pipe former to latter. For example, streaming to [vlc](https://www.videolan.org/) can be achieved with:
youtube-dl -o - "http://www.youtube.com/watch?v=BaW_jenozKcj" | vlc -
youtube-dl -o - "https://www.youtube.com/watch?v=BaW_jenozKcj" | vlc -
### How do I download only new videos from a playlist?
@ -869,7 +909,7 @@ When youtube-dl detects an HLS video, it can download it either with the built-i
When youtube-dl knows that one particular downloader works better for a given website, that downloader will be picked. Otherwise, youtube-dl will pick the best downloader for general compatibility, which at the moment happens to be ffmpeg. This choice may change in future versions of youtube-dl, with improvements of the built-in downloader and/or ffmpeg.
In particular, the generic extractor (used when your website is not in the [list of supported sites by youtube-dl](http://rg3.github.io/youtube-dl/supportedsites.html) cannot mandate one specific downloader.
In particular, the generic extractor (used when your website is not in the [list of supported sites by youtube-dl](https://ytdl-org.github.io/youtube-dl/supportedsites.html) cannot mandate one specific downloader.
If you put either `--hls-prefer-native` or `--hls-prefer-ffmpeg` into your configuration, a different subset of videos will fail to download correctly. Instead, it is much better to [file an issue](https://yt-dl.org/bug) or a pull request which details why the native or the ffmpeg HLS downloader is a better choice for your use case.
@ -895,7 +935,7 @@ Feel free to bump the issue from time to time by writing a small comment ("Issue
### How can I detect whether a given URL is supported by youtube-dl?
For one, have a look at the [list of supported sites](docs/supportedsites.md). Note that it can sometimes happen that the site changes its URL scheme (say, from http://example.com/video/1234567 to http://example.com/v/1234567 ) and youtube-dl reports an URL of a service in that list as unsupported. In that case, simply report a bug.
For one, have a look at the [list of supported sites](docs/supportedsites.md). Note that it can sometimes happen that the site changes its URL scheme (say, from https://example.com/video/1234567 to https://example.com/v/1234567 ) and youtube-dl reports an URL of a service in that list as unsupported. In that case, simply report a bug.
It is *not* possible to detect whether a URL is supported or not. That's because youtube-dl contains a generic extractor which matches **all** URLs. You may be tempted to disable, exclude, or remove the generic extractor, but the generic extractor not only allows users to extract videos from lots of websites that embed a video from another service, but may also be used to extract video from a service that it's hosting itself. Therefore, we neither recommend nor support disabling, excluding, or removing the generic extractor.
@ -909,7 +949,7 @@ youtube-dl is an open-source project manned by too few volunteers, so we'd rathe
# DEVELOPER INSTRUCTIONS
Most users do not need to build youtube-dl and can [download the builds](http://rg3.github.io/youtube-dl/download.html) or get them from their distribution.
Most users do not need to build youtube-dl and can [download the builds](https://ytdl-org.github.io/youtube-dl/download.html) or get them from their distribution.
To run youtube-dl as a developer, you don't need to build anything either. Simply execute
@ -921,6 +961,8 @@ To run the test, simply invoke your favorite test runner, or execute a test file
python test/test_download.py
nosetests
See item 6 of [new extractor tutorial](#adding-support-for-a-new-site) for how to run extractor specific test cases.
If you want to create a build of youtube-dl yourself, you'll need
* python
@ -935,7 +977,7 @@ If you want to add support for a new site, first of all **make sure** this site
After you have ensured this site is distributing its content legally, you can follow this quick list (assuming your service is called `yourextractor`):
1. [Fork this repository](https://github.com/rg3/youtube-dl/fork)
1. [Fork this repository](https://github.com/ytdl-org/youtube-dl/fork)
2. Check out the source code with:
git clone git@github.com:YOUR_GITHUB_USERNAME/youtube-dl.git
@ -957,7 +999,7 @@ After you have ensured this site is distributing its content legally, you can fo
class YourExtractorIE(InfoExtractor):
_VALID_URL = r'https?://(?:www\.)?yourextractor\.com/watch/(?P<id>[0-9]+)'
_TEST = {
'url': 'http://yourextractor.com/watch/42',
'url': 'https://yourextractor.com/watch/42',
'md5': 'TODO: md5 sum of the first 10241 bytes of the video file (use --test)',
'info_dict': {
'id': '42',
@ -987,18 +1029,22 @@ After you have ensured this site is distributing its content legally, you can fo
# TODO more properties (see youtube_dl/extractor/common.py)
}
```
5. Add an import in [`youtube_dl/extractor/extractors.py`](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/extractors.py).
6. Run `python test/test_download.py TestDownload.test_YourExtractor`. This *should fail* at first, but you can continually re-run it until you're done. If you decide to add more than one test, then rename ``_TEST`` to ``_TESTS`` and make it into a list of dictionaries. The tests will then be named `TestDownload.test_YourExtractor`, `TestDownload.test_YourExtractor_1`, `TestDownload.test_YourExtractor_2`, etc.
7. Have a look at [`youtube_dl/extractor/common.py`](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/common.py) for possible helper methods and a [detailed description of what your extractor should and may return](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/common.py#L74-L252). Add tests and code for as many as you want.
8. Make sure your code follows [youtube-dl coding conventions](#youtube-dl-coding-conventions) and check the code with [flake8](https://pypi.python.org/pypi/flake8). Also make sure your code works under all [Python](http://www.python.org/) versions claimed supported by youtube-dl, namely 2.6, 2.7, and 3.2+.
9. When the tests pass, [add](http://git-scm.com/docs/git-add) the new files and [commit](http://git-scm.com/docs/git-commit) them and [push](http://git-scm.com/docs/git-push) the result, like this:
5. Add an import in [`youtube_dl/extractor/extractors.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/extractor/extractors.py).
6. Run `python test/test_download.py TestDownload.test_YourExtractor`. This *should fail* at first, but you can continually re-run it until you're done. If you decide to add more than one test, then rename ``_TEST`` to ``_TESTS`` and make it into a list of dictionaries. The tests will then be named `TestDownload.test_YourExtractor`, `TestDownload.test_YourExtractor_1`, `TestDownload.test_YourExtractor_2`, etc. Note that tests with `only_matching` key in test's dict are not counted in.
7. Have a look at [`youtube_dl/extractor/common.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/extractor/common.py) for possible helper methods and a [detailed description of what your extractor should and may return](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303). Add tests and code for as many as you want.
8. Make sure your code follows [youtube-dl coding conventions](#youtube-dl-coding-conventions) and check the code with [flake8](https://flake8.pycqa.org/en/latest/index.html#quickstart):
$ flake8 youtube_dl/extractor/yourextractor.py
9. Make sure your code works under all [Python](https://www.python.org/) versions claimed supported by youtube-dl, namely 2.6, 2.7, and 3.2+.
10. When the tests pass, [add](https://git-scm.com/docs/git-add) the new files and [commit](https://git-scm.com/docs/git-commit) them and [push](https://git-scm.com/docs/git-push) the result, like this:
$ git add youtube_dl/extractor/extractors.py
$ git add youtube_dl/extractor/yourextractor.py
$ git commit -m '[yourextractor] Add new extractor'
$ git push origin yourextractor
10. Finally, [create a pull request](https://help.github.com/articles/creating-a-pull-request). We'll then review and merge it.
11. Finally, [create a pull request](https://help.github.com/articles/creating-a-pull-request). We'll then review and merge it.
In any case, thank you very much for your contributions!
@ -1010,7 +1056,7 @@ Extractors are very fragile by nature since they depend on the layout of the sou
### Mandatory and optional metafields
For extraction to work youtube-dl relies on metadata your extractor extracts and provides to youtube-dl expressed by an [information dictionary](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/common.py#L75-L257) or simply *info dict*. Only the following meta fields in the *info dict* are considered mandatory for a successful extraction process by youtube-dl:
For extraction to work youtube-dl relies on metadata your extractor extracts and provides to youtube-dl expressed by an [information dictionary](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303) or simply *info dict*. Only the following meta fields in the *info dict* are considered mandatory for a successful extraction process by youtube-dl:
- `id` (media identifier)
- `title` (media title)
@ -1018,7 +1064,7 @@ For extraction to work youtube-dl relies on metadata your extractor extracts and
In fact only the last option is technically mandatory (i.e. if you can't figure out the download location of the media the extraction does not make any sense). But by convention youtube-dl also treats `id` and `title` as mandatory. Thus the aforementioned metafields are the critical data that the extraction does not make any sense without and if any of them fail to be extracted then the extractor is considered completely broken.
[Any field](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/common.py#L149-L257) apart from the aforementioned ones are considered **optional**. That means that extraction should be **tolerant** to situations when sources for these fields can potentially be unavailable (even if they are always available at the moment) and **future-proof** in order not to break the extraction of general purpose mandatory fields.
[Any field](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L188-L303) apart from the aforementioned ones are considered **optional**. That means that extraction should be **tolerant** to situations when sources for these fields can potentially be unavailable (even if they are always available at the moment) and **future-proof** in order not to break the extraction of general purpose mandatory fields.
#### Example
@ -1094,11 +1140,33 @@ title = meta.get('title') or self._og_search_title(webpage)
This code will try to extract from `meta` first and if it fails it will try extracting `og:title` from a `webpage`.
### Make regular expressions flexible
### Regular expressions
When using regular expressions try to write them fuzzy and flexible.
#### Don't capture groups you don't use
#### Example
Capturing group must be an indication that it's used somewhere in the code. Any group that is not used must be non capturing.
##### Example
Don't capture id attribute name here since you can't use it for anything anyway.
Correct:
```python
r'(?:id|ID)=(?P<id>\d+)'
```
Incorrect:
```python
r'(id|ID)=(?P<id>\d+)'
```
#### Make regular expressions relaxed and flexible
When using regular expressions try to write them fuzzy, relaxed and flexible, skipping insignificant parts that are more likely to change, allowing both single and double quotes for quoted values and so on.
##### Example
Say you need to extract `title` from the following HTML code:
@ -1131,13 +1199,121 @@ title = self._search_regex(
webpage, 'title', group='title')
```
### Use safe conversion functions
### Long lines policy
Wrap all extracted numeric data into safe functions from `utils`: `int_or_none`, `float_or_none`. Use them for string to number conversions as well.
There is a soft limit to keep lines of code under 80 characters long. This means it should be respected if possible and if it does not make readability and code maintenance worse.
For example, you should **never** split long string literals like URLs or some other often copied entities over multiple lines to fit this limit:
Correct:
```python
'https://www.youtube.com/watch?v=FqZTN594JQw&list=PLMYEtVRpaqY00V9W81Cwmzp6N6vZqfUKD4'
```
Incorrect:
```python
'https://www.youtube.com/watch?v=FqZTN594JQw&list='
'PLMYEtVRpaqY00V9W81Cwmzp6N6vZqfUKD4'
```
### Inline values
Extracting variables is acceptable for reducing code duplication and improving readability of complex expressions. However, you should avoid extracting variables used only once and moving them to opposite parts of the extractor file, which makes reading the linear flow difficult.
#### Example
Correct:
```python
title = self._html_search_regex(r'<title>([^<]+)</title>', webpage, 'title')
```
Incorrect:
```python
TITLE_RE = r'<title>([^<]+)</title>'
# ...some lines of code...
title = self._html_search_regex(TITLE_RE, webpage, 'title')
```
### Collapse fallbacks
Multiple fallback values can quickly become unwieldy. Collapse multiple fallback values into a single expression via a list of patterns.
#### Example
Good:
```python
description = self._html_search_meta(
['og:description', 'description', 'twitter:description'],
webpage, 'description', default=None)
```
Unwieldy:
```python
description = (
self._og_search_description(webpage, default=None)
or self._html_search_meta('description', webpage, default=None)
or self._html_search_meta('twitter:description', webpage, default=None))
```
Methods supporting list of patterns are: `_search_regex`, `_html_search_regex`, `_og_search_property`, `_html_search_meta`.
### Trailing parentheses
Always move trailing parentheses after the last argument.
#### Example
Correct:
```python
lambda x: x['ResultSet']['Result'][0]['VideoUrlSet']['VideoUrl'],
list)
```
Incorrect:
```python
lambda x: x['ResultSet']['Result'][0]['VideoUrlSet']['VideoUrl'],
list,
)
```
### Use convenience conversion and parsing functions
Wrap all extracted numeric data into safe functions from [`youtube_dl/utils.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/utils.py): `int_or_none`, `float_or_none`. Use them for string to number conversions as well.
Use `url_or_none` for safe URL processing.
Use `try_get` for safe metadata extraction from parsed JSON.
Use `unified_strdate` for uniform `upload_date` or any `YYYYMMDD` meta field extraction, `unified_timestamp` for uniform `timestamp` extraction, `parse_filesize` for `filesize` extraction, `parse_count` for count meta fields extraction, `parse_resolution`, `parse_duration` for `duration` extraction, `parse_age_limit` for `age_limit` extraction.
Explore [`youtube_dl/utils.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/utils.py) for more useful convenience functions.
#### More examples
##### Safely extract optional description from parsed JSON
```python
description = try_get(response, lambda x: x['result']['video'][0]['summary'], compat_str)
```
##### Safely extract more optional metadata
```python
video = try_get(response, lambda x: x['result']['video'][0], dict) or {}
description = video.get('summary')
duration = float_or_none(video.get('durationMs'), scale=1000)
view_count = int_or_none(video.get('views'))
```
# EMBEDDING YOUTUBE-DL
youtube-dl makes the best effort to be a good command-line program, and thus should be callable from any programming language. If you encounter any problems parsing its output, feel free to [create a report](https://github.com/rg3/youtube-dl/issues/new).
youtube-dl makes the best effort to be a good command-line program, and thus should be callable from any programming language. If you encounter any problems parsing its output, feel free to [create a report](https://github.com/ytdl-org/youtube-dl/issues/new).
From a Python program, you can embed youtube-dl in a more powerful fashion, like this:
@ -1147,10 +1323,10 @@ import youtube_dl
ydl_opts = {}
with youtube_dl.YoutubeDL(ydl_opts) as ydl:
ydl.download(['http://www.youtube.com/watch?v=BaW_jenozKc'])
ydl.download(['https://www.youtube.com/watch?v=BaW_jenozKc'])
```
Most likely, you'll want to use various options. For a list of options available, have a look at [`youtube_dl/YoutubeDL.py`](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/YoutubeDL.py#L129-L279). For a start, if you want to intercept youtube-dl's output, set a `logger` object.
Most likely, you'll want to use various options. For a list of options available, have a look at [`youtube_dl/YoutubeDL.py`](https://github.com/ytdl-org/youtube-dl/blob/3e4cedf9e8cd3157df2457df7274d0c842421945/youtube_dl/YoutubeDL.py#L137-L312). For a start, if you want to intercept youtube-dl's output, set a `logger` object.
Here's a more complete example of a program that outputs only errors (and a short message after the download is finished), and downloads/converts the video to an mp3 file:
@ -1186,19 +1362,19 @@ ydl_opts = {
'progress_hooks': [my_hook],
}
with youtube_dl.YoutubeDL(ydl_opts) as ydl:
ydl.download(['http://www.youtube.com/watch?v=BaW_jenozKc'])
ydl.download(['https://www.youtube.com/watch?v=BaW_jenozKc'])
```
# BUGS
Bugs and suggestions should be reported at: <https://github.com/rg3/youtube-dl/issues>. Unless you were prompted to or there is another pertinent reason (e.g. GitHub fails to accept the bug report), please do not send bug reports via personal email. For discussions, join us in the IRC channel [#youtube-dl](irc://chat.freenode.net/#youtube-dl) on freenode ([webchat](http://webchat.freenode.net/?randomnick=1&channels=youtube-dl)).
Bugs and suggestions should be reported at: <https://github.com/ytdl-org/youtube-dl/issues>. Unless you were prompted to or there is another pertinent reason (e.g. GitHub fails to accept the bug report), please do not send bug reports via personal email. For discussions, join us in the IRC channel [#youtube-dl](irc://chat.freenode.net/#youtube-dl) on freenode ([webchat](https://webchat.freenode.net/?randomnick=1&channels=youtube-dl)).
**Please include the full output of youtube-dl when run with `-v`**, i.e. **add** `-v` flag to **your command line**, copy the **whole** output and post it in the issue body wrapped in \`\`\` for better formatting. It should look similar to this:
```
$ youtube-dl -v <your command line>
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version 2015.12.06
[debug] Git HEAD: 135392e
@ -1229,7 +1405,7 @@ For bug reports, this means that your report should contain the *complete* outpu
If your server has multiple IPs or you suspect censorship, adding `--call-home` may be a good idea to get more diagnostics. If the error is `ERROR: Unable to extract ...` and you cannot reproduce it from multiple countries, add `--dump-pages` (warning: this will yield a rather large output, redirect it to the file `log.txt` by adding `>log.txt 2>&1` to your command-line) or upload the `.dump` files you get when you add `--write-pages` [somewhere](https://gist.github.com/).
**Site support requests must contain an example URL**. An example URL is a URL you might want to download, like `http://www.youtube.com/watch?v=BaW_jenozKc`. There should be an obvious video present. Except under very special circumstances, the main page of a video service (e.g. `http://www.youtube.com/`) is *not* an example URL.
**Site support requests must contain an example URL**. An example URL is a URL you might want to download, like `https://www.youtube.com/watch?v=BaW_jenozKc`. There should be an obvious video present. Except under very special circumstances, the main page of a video service (e.g. `https://www.youtube.com/`) is *not* an example URL.
### Are you using the latest version?
@ -1237,11 +1413,11 @@ Before reporting any issue, type `youtube-dl -U`. This should report that you're
### Is the issue already documented?
Make sure that someone has not already opened the issue you're trying to open. Search at the top of the window or browse the [GitHub Issues](https://github.com/rg3/youtube-dl/search?type=Issues) of this repository. If there is an issue, feel free to write something along the lines of "This affects me as well, with version 2015.01.01. Here is some more information on the issue: ...". While some issues may be old, a new post into them often spurs rapid activity.
Make sure that someone has not already opened the issue you're trying to open. Search at the top of the window or browse the [GitHub Issues](https://github.com/ytdl-org/youtube-dl/search?type=Issues) of this repository. If there is an issue, feel free to write something along the lines of "This affects me as well, with version 2015.01.01. Here is some more information on the issue: ...". While some issues may be old, a new post into them often spurs rapid activity.
### Why are existing options not enough?
Before requesting a new feature, please have a quick peek at [the list of supported options](https://github.com/rg3/youtube-dl/blob/master/README.md#options). Many feature requests are for features that actually exist already! Please, absolutely do show off your work in the issue report and detail how the existing similar options do *not* solve your problem.
Before requesting a new feature, please have a quick peek at [the list of supported options](https://github.com/ytdl-org/youtube-dl/blob/master/README.md#options). Many feature requests are for features that actually exist already! Please, absolutely do show off your work in the issue report and detail how the existing similar options do *not* solve your problem.
### Is there enough context in your bug report?

View File

@ -322,7 +322,7 @@ class GITBuilder(GITInfoBuilder):
class YoutubeDLBuilder(object):
authorizedUsers = ['fraca7', 'phihag', 'rg3', 'FiloSottile']
authorizedUsers = ['fraca7', 'phihag', 'rg3', 'FiloSottile', 'ytdl-org']
def __init__(self, **kwargs):
if self.repoName != 'youtube-dl':

View File

@ -14,7 +14,7 @@ import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from test.helper import get_testcases
from test.helper import gettestcases
from youtube_dl.utils import compat_urllib_parse_urlparse
from youtube_dl.utils import compat_urllib_request
@ -24,7 +24,7 @@ if len(sys.argv) > 1:
else:
METHOD = 'EURISTIC'
for test in get_testcases():
for test in gettestcases():
if METHOD == 'EURISTIC':
try:
webpage = compat_urllib_request.urlopen(test['url'], timeout=10).read()
@ -45,12 +45,12 @@ for test in get_testcases():
RESULT = ('.' + domain + '\n' in LIST or '\n' + domain + '\n' in LIST)
if RESULT and ('info_dict' not in test or 'age_limit' not in test['info_dict'] or
test['info_dict']['age_limit'] != 18):
if RESULT and ('info_dict' not in test or 'age_limit' not in test['info_dict']
or test['info_dict']['age_limit'] != 18):
print('\nPotential missing age_limit check: {0}'.format(test['name']))
elif not RESULT and ('info_dict' in test and 'age_limit' in test['info_dict'] and
test['info_dict']['age_limit'] == 18):
elif not RESULT and ('info_dict' in test and 'age_limit' in test['info_dict']
and test['info_dict']['age_limit'] == 18):
print('\nPotential false negative: {0}'.format(test['name']))
else:

View File

@ -1,7 +1,6 @@
#!/usr/bin/env python
from __future__ import unicode_literals
import base64
import io
import json
import mimetypes
@ -15,7 +14,6 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from youtube_dl.compat import (
compat_basestring,
compat_input,
compat_getpass,
compat_print,
compat_urllib_request,
@ -27,8 +25,8 @@ from youtube_dl.utils import (
class GitHubReleaser(object):
_API_URL = 'https://api.github.com/repos/rg3/youtube-dl/releases'
_UPLOADS_URL = 'https://uploads.github.com/repos/rg3/youtube-dl/releases/%s/assets?name=%s'
_API_URL = 'https://api.github.com/repos/ytdl-org/youtube-dl/releases'
_UPLOADS_URL = 'https://uploads.github.com/repos/ytdl-org/youtube-dl/releases/%s/assets?name=%s'
_NETRC_MACHINE = 'github.com'
def __init__(self, debuglevel=0):
@ -40,28 +38,20 @@ class GitHubReleaser(object):
try:
info = netrc.netrc().authenticators(self._NETRC_MACHINE)
if info is not None:
self._username = info[0]
self._password = info[2]
self._token = info[2]
compat_print('Using GitHub credentials found in .netrc...')
return
else:
compat_print('No GitHub credentials found in .netrc')
except (IOError, netrc.NetrcParseError):
compat_print('Unable to parse .netrc')
self._username = compat_input(
'Type your GitHub username or email address and press [Return]: ')
self._password = compat_getpass(
'Type your GitHub password and press [Return]: ')
self._token = compat_getpass(
'Type your GitHub PAT (personal access token) and press [Return]: ')
def _call(self, req):
if isinstance(req, compat_basestring):
req = sanitized_Request(req)
# Authorizing manually since GitHub does not response with 401 with
# WWW-Authenticate header set (see
# https://developer.github.com/v3/#basic-authentication)
b64 = base64.b64encode(
('%s:%s' % (self._username, self._password)).encode('utf-8')).decode('ascii')
req.add_header('Authorization', 'Basic %s' % b64)
req.add_header('Authorization', 'token %s' % self._token)
response = self._opener.open(req).read().decode('utf-8')
return json.loads(response)

View File

@ -1,27 +1,22 @@
#!/usr/bin/env python3
from __future__ import unicode_literals
import hashlib
import urllib.request
import json
versions_info = json.load(open('update/versions.json'))
version = versions_info['latest']
URL = versions_info['versions'][version]['bin'][0]
data = urllib.request.urlopen(URL).read()
version_dict = versions_info['versions'][version]
# Read template page
with open('download.html.in', 'r', encoding='utf-8') as tmplf:
template = tmplf.read()
sha256sum = hashlib.sha256(data).hexdigest()
template = template.replace('@PROGRAM_VERSION@', version)
template = template.replace('@PROGRAM_URL@', URL)
template = template.replace('@PROGRAM_SHA256SUM@', sha256sum)
template = template.replace('@EXE_URL@', versions_info['versions'][version]['exe'][0])
template = template.replace('@EXE_SHA256SUM@', versions_info['versions'][version]['exe'][1])
template = template.replace('@TAR_URL@', versions_info['versions'][version]['tar'][0])
template = template.replace('@TAR_SHA256SUM@', versions_info['versions'][version]['tar'][1])
template = template.replace('@PROGRAM_URL@', version_dict['bin'][0])
template = template.replace('@PROGRAM_SHA256SUM@', version_dict['bin'][1])
template = template.replace('@EXE_URL@', version_dict['exe'][0])
template = template.replace('@EXE_SHA256SUM@', version_dict['exe'][1])
template = template.replace('@TAR_URL@', version_dict['tar'][0])
template = template.replace('@TAR_SHA256SUM@', version_dict['tar'][1])
with open('download.html', 'w', encoding='utf-8') as dlf:
dlf.write(template)

View File

@ -13,7 +13,7 @@ year = str(datetime.datetime.now().year)
for fn in glob.glob('*.html*'):
with io.open(fn, encoding='utf-8') as f:
content = f.read()
newc = re.sub(r'(?P<copyright>Copyright © 2006-)(?P<year>[0-9]{4})', 'Copyright © 2006-' + year, content)
newc = re.sub(r'(?P<copyright>Copyright © 2011-)(?P<year>[0-9]{4})', 'Copyright © 2011-' + year, content)
if content != newc:
tmpFn = fn + '.part'
with io.open(tmpFn, 'wt', encoding='utf-8') as outf:

View File

@ -10,7 +10,7 @@ import textwrap
atom_template = textwrap.dedent("""\
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<link rel="self" href="http://rg3.github.io/youtube-dl/update/releases.atom" />
<link rel="self" href="http://ytdl-org.github.io/youtube-dl/update/releases.atom" />
<title>youtube-dl releases</title>
<id>https://yt-dl.org/feed/youtube-dl-updates-feed</id>
<updated>@TIMESTAMP@</updated>
@ -21,7 +21,7 @@ entry_template = textwrap.dedent("""
<entry>
<id>https://yt-dl.org/feed/youtube-dl-updates-feed/youtube-dl-@VERSION@</id>
<title>New version @VERSION@</title>
<link href="http://rg3.github.io/youtube-dl" />
<link href="http://ytdl-org.github.io/youtube-dl" />
<content type="xhtml">
<div xmlns="http://www.w3.org/1999/xhtml">
Downloads available at <a href="https://yt-dl.org/downloads/@VERSION@/">https://yt-dl.org/downloads/@VERSION@/</a>

5
devscripts/install_jython.sh Executable file
View File

@ -0,0 +1,5 @@
#!/bin/bash
wget http://central.maven.org/maven2/org/python/jython-installer/2.7.1/jython-installer-2.7.1.jar
java -jar jython-installer-2.7.1.jar -s -d "$HOME/jython"
$HOME/jython/bin/jython -m pip install nose

View File

@ -1,6 +1,7 @@
from __future__ import unicode_literals, print_function
from inspect import getsource
import io
import os
from os.path import dirname as dirn
import sys
@ -60,7 +61,7 @@ def build_lazy_ie(ie, name):
return s
# find the correct sorting and add the required base classes so that sublcasses
# find the correct sorting and add the required base classes so that subclasses
# can be correctly created
classes = _ALL_CLASSES[:-1]
ordered_cls = []
@ -95,5 +96,5 @@ module_contents.append(
module_src = '\n'.join(module_contents) + '\n'
with open(lazy_extractors_filename, 'wt') as f:
with io.open(lazy_extractors_filename, 'wt', encoding='utf-8') as f:
f.write(module_src)

View File

@ -8,7 +8,7 @@ import re
ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
README_FILE = os.path.join(ROOT_DIR, 'README.md')
PREFIX = '''%YOUTUBE-DL(1)
PREFIX = r'''%YOUTUBE-DL(1)
# NAME

View File

@ -78,8 +78,8 @@ sed -i "s/__version__ = '.*'/__version__ = '$version'/" youtube_dl/version.py
sed -i "s/<unreleased>/$version/" ChangeLog
/bin/echo -e "\n### Committing documentation, templates and youtube_dl/version.py..."
make README.md CONTRIBUTING.md .github/ISSUE_TEMPLATE.md supportedsites
git add README.md CONTRIBUTING.md .github/ISSUE_TEMPLATE.md docs/supportedsites.md youtube_dl/version.py ChangeLog
make README.md CONTRIBUTING.md issuetemplates supportedsites
git add README.md CONTRIBUTING.md .github/ISSUE_TEMPLATE/1_broken_site.md .github/ISSUE_TEMPLATE/2_site_support_request.md .github/ISSUE_TEMPLATE/3_site_feature_request.md .github/ISSUE_TEMPLATE/4_bug_report.md .github/ISSUE_TEMPLATE/5_feature_request.md .github/ISSUE_TEMPLATE/6_question.md docs/supportedsites.md youtube_dl/version.py ChangeLog
git commit $gpg_sign_commits -m "release $version"
/bin/echo -e "\n### Now tagging, signing and pushing..."
@ -96,7 +96,7 @@ git push origin "$version"
REV=$(git rev-parse HEAD)
make youtube-dl youtube-dl.tar.gz
read -p "VM running? (y/n) " -n 1
wget "http://$buildserver/build/rg3/youtube-dl/youtube-dl.exe?rev=$REV" -O youtube-dl.exe
wget "http://$buildserver/build/ytdl-org/youtube-dl/youtube-dl.exe?rev=$REV" -O youtube-dl.exe
mkdir -p "build/$version"
mv youtube-dl youtube-dl.exe "build/$version"
mv youtube-dl.tar.gz "build/$version/youtube-dl-$version.tar.gz"

7
devscripts/run_tests.sh Normal file → Executable file
View File

@ -1,8 +1,10 @@
#!/bin/bash
DOWNLOAD_TESTS="age_restriction|download|subtitles|write_annotations|iqiyi_sdk_interpreter"
# Keep this list in sync with the `offlinetest` target in Makefile
DOWNLOAD_TESTS="age_restriction|download|iqiyi_sdk_interpreter|socks|subtitles|write_annotations|youtube_lists|youtube_signature"
test_set=""
multiprocess_args=""
case "$YTDL_TEST_SET" in
core)
@ -10,10 +12,11 @@ case "$YTDL_TEST_SET" in
;;
download)
test_set="-I test_(?!$DOWNLOAD_TESTS).+\.py"
multiprocess_args="--processes=4 --process-timeout=540"
;;
*)
break
;;
esac
nosetests test --verbose $test_set
nosetests test --verbose $test_set $multiprocess_args

View File

@ -24,7 +24,7 @@ total_bytes = 0
for page in itertools.count(1):
releases = json.loads(compat_urllib_request.urlopen(
'https://api.github.com/repos/rg3/youtube-dl/releases?page=%s' % page
'https://api.github.com/repos/ytdl-org/youtube-dl/releases?page=%s' % page
).read().decode('utf-8'))
if not releases:

File diff suppressed because it is too large Load Diff

View File

@ -2,5 +2,5 @@
universal = True
[flake8]
exclude = youtube_dl/extractor/__init__.py,devscripts/buildserver.py,devscripts/lazy_load_template.py,devscripts/make_issue_template.py,setup.py,build,.git
ignore = E402,E501,E731
exclude = youtube_dl/extractor/__init__.py,devscripts/buildserver.py,devscripts/lazy_load_template.py,devscripts/make_issue_template.py,setup.py,build,.git,venv
ignore = E402,E501,E731,E741,W503

View File

@ -104,11 +104,12 @@ setup(
version=__version__,
description=DESCRIPTION,
long_description=LONG_DESCRIPTION,
url='https://github.com/rg3/youtube-dl',
url='https://github.com/ytdl-org/youtube-dl',
author='Ricardo Garcia',
author_email='ytdl@yt-dl.org',
maintainer='Philipp Hagemeister',
maintainer_email='phihag@phihag.de',
maintainer='Sergey M.',
maintainer_email='dstftw@gmail.com',
license='Unlicense',
packages=[
'youtube_dl',
'youtube_dl.extractor', 'youtube_dl.downloader',
@ -123,6 +124,8 @@ setup(
'Development Status :: 5 - Production/Stable',
'Environment :: Console',
'License :: Public Domain',
'Programming Language :: Python',
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.6',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
@ -130,6 +133,14 @@ setup(
'Programming Language :: Python :: 3.3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: Implementation',
'Programming Language :: Python :: Implementation :: CPython',
'Programming Language :: Python :: Implementation :: IronPython',
'Programming Language :: Python :: Implementation :: Jython',
'Programming Language :: Python :: Implementation :: PyPy',
],
cmdclass={'build_lazy_extractors': build_lazy_extractors},

View File

@ -7,6 +7,7 @@ import json
import os.path
import re
import types
import ssl
import sys
import youtube_dl.extractor
@ -152,15 +153,27 @@ def expect_value(self, got, expected, field):
isinstance(got, compat_str),
'Expected field %s to be a unicode object, but got value %r of type %r' % (field, got, type(got)))
got = 'md5:' + md5(got)
elif isinstance(expected, compat_str) and expected.startswith('mincount:'):
elif isinstance(expected, compat_str) and re.match(r'^(?:min|max)?count:\d+', expected):
self.assertTrue(
isinstance(got, (list, dict)),
'Expected field %s to be a list or a dict, but it is of type %s' % (
field, type(got).__name__))
expected_num = int(expected.partition(':')[2])
assertGreaterEqual(
op, _, expected_num = expected.partition(':')
expected_num = int(expected_num)
if op == 'mincount':
assert_func = assertGreaterEqual
msg_tmpl = 'Expected %d items in field %s, but only got %d'
elif op == 'maxcount':
assert_func = assertLessEqual
msg_tmpl = 'Expected maximum %d items in field %s, but got %d'
elif op == 'count':
assert_func = assertEqual
msg_tmpl = 'Expected exactly %d items in field %s, but got %d'
else:
assert False
assert_func(
self, len(got), expected_num,
'Expected %d items in field %s, but only got %d' % (expected_num, field, len(got)))
msg_tmpl % (expected_num, field, len(got)))
return
self.assertEqual(
expected, got,
@ -236,6 +249,20 @@ def assertGreaterEqual(self, got, expected, msg=None):
self.assertTrue(got >= expected, msg)
def assertLessEqual(self, got, expected, msg=None):
if not (got <= expected):
if msg is None:
msg = '%r not less than or equal to %r' % (got, expected)
self.assertTrue(got <= expected, msg)
def assertEqual(self, got, expected, msg=None):
if not (got == expected):
if msg is None:
msg = '%r not equal to %r' % (got, expected)
self.assertTrue(got == expected, msg)
def expect_warnings(ydl, warnings_re):
real_warning = ydl.report_warning
@ -244,3 +271,12 @@ def expect_warnings(ydl, warnings_re):
real_warning(w)
ydl.report_warning = _report_warning
def http_server_port(httpd):
if os.name == 'java' and isinstance(httpd.socket, ssl.SSLSocket):
# In Jython SSLSocket is not a subclass of socket.socket
sock = httpd.socket.sock
else:
sock = httpd.socket
return sock.getsockname()[1]

View File

@ -37,7 +37,7 @@
"writeinfojson": true,
"writesubtitles": false,
"allsubtitles": false,
"listssubtitles": false,
"listsubtitles": false,
"socket_timeout": 20,
"fixup": "never"
}

File diff suppressed because it is too large Load Diff

View File

@ -1,4 +1,5 @@
#!/usr/bin/env python
# coding: utf-8
from __future__ import unicode_literals
@ -40,6 +41,7 @@ def _make_result(formats, **kwargs):
'id': 'testid',
'title': 'testttitle',
'extractor': 'testex',
'extractor_key': 'TestEx',
}
res.update(**kwargs)
return res
@ -237,6 +239,76 @@ class TestFormatSelection(unittest.TestCase):
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'vid-vcodec-dot')
def test_format_selection_string_ops(self):
formats = [
{'format_id': 'abc-cba', 'ext': 'mp4', 'url': TEST_URL},
{'format_id': 'zxc-cxz', 'ext': 'webm', 'url': TEST_URL},
]
info_dict = _make_result(formats)
# equals (=)
ydl = YDL({'format': '[format_id=abc-cba]'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'abc-cba')
# does not equal (!=)
ydl = YDL({'format': '[format_id!=abc-cba]'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'zxc-cxz')
ydl = YDL({'format': '[format_id!=abc-cba][format_id!=zxc-cxz]'})
self.assertRaises(ExtractorError, ydl.process_ie_result, info_dict.copy())
# starts with (^=)
ydl = YDL({'format': '[format_id^=abc]'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'abc-cba')
# does not start with (!^=)
ydl = YDL({'format': '[format_id!^=abc]'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'zxc-cxz')
ydl = YDL({'format': '[format_id!^=abc][format_id!^=zxc]'})
self.assertRaises(ExtractorError, ydl.process_ie_result, info_dict.copy())
# ends with ($=)
ydl = YDL({'format': '[format_id$=cba]'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'abc-cba')
# does not end with (!$=)
ydl = YDL({'format': '[format_id!$=cba]'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'zxc-cxz')
ydl = YDL({'format': '[format_id!$=cba][format_id!$=cxz]'})
self.assertRaises(ExtractorError, ydl.process_ie_result, info_dict.copy())
# contains (*=)
ydl = YDL({'format': '[format_id*=bc-cb]'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'abc-cba')
# does not contain (!*=)
ydl = YDL({'format': '[format_id!*=bc-cb]'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'zxc-cxz')
ydl = YDL({'format': '[format_id!*=abc][format_id!*=zxc]'})
self.assertRaises(ExtractorError, ydl.process_ie_result, info_dict.copy())
ydl = YDL({'format': '[format_id!*=-]'})
self.assertRaises(ExtractorError, ydl.process_ie_result, info_dict.copy())
def test_youtube_format_selection(self):
order = [
'38', '37', '46', '22', '45', '35', '44', '18', '34', '43', '6', '5', '17', '36', '13',
@ -339,7 +411,7 @@ class TestFormatSelection(unittest.TestCase):
# For extractors with incomplete formats (all formats are audio-only or
# video-only) best and worst should fallback to corresponding best/worst
# video-only or audio-only formats (as per
# https://github.com/rg3/youtube-dl/pull/5556)
# https://github.com/ytdl-org/youtube-dl/pull/5556)
formats = [
{'format_id': 'low', 'ext': 'mp3', 'preference': 1, 'vcodec': 'none', 'url': TEST_URL},
{'format_id': 'high', 'ext': 'mp3', 'preference': 2, 'vcodec': 'none', 'url': TEST_URL},
@ -369,6 +441,19 @@ class TestFormatSelection(unittest.TestCase):
ydl = YDL({'format': 'best[height>360]'})
self.assertRaises(ExtractorError, ydl.process_ie_result, info_dict.copy())
def test_format_selection_issue_10083(self):
# See https://github.com/ytdl-org/youtube-dl/issues/10083
formats = [
{'format_id': 'regular', 'height': 360, 'url': TEST_URL},
{'format_id': 'video', 'height': 720, 'acodec': 'none', 'url': TEST_URL},
{'format_id': 'audio', 'vcodec': 'none', 'url': TEST_URL},
]
info_dict = _make_result(formats)
ydl = YDL({'format': 'best[height>360]/bestvideo[height>360]+bestaudio'})
ydl.process_ie_result(info_dict.copy())
self.assertEqual(ydl.downloaded_info_dicts[0]['format_id'], 'video+audio')
def test_invalid_format_specs(self):
def assert_syntax_error(format_spec):
ydl = YDL({'format': format_spec})
@ -447,6 +532,23 @@ class TestFormatSelection(unittest.TestCase):
pass
self.assertEqual(ydl.downloaded_info_dicts, [])
def test_default_format_spec(self):
ydl = YDL({'simulate': True})
self.assertEqual(ydl._default_format_spec({}), 'bestvideo+bestaudio/best')
ydl = YDL({})
self.assertEqual(ydl._default_format_spec({'is_live': True}), 'best/bestvideo+bestaudio')
ydl = YDL({'simulate': True})
self.assertEqual(ydl._default_format_spec({'is_live': True}), 'bestvideo+bestaudio/best')
ydl = YDL({'outtmpl': '-'})
self.assertEqual(ydl._default_format_spec({}), 'best/bestvideo+bestaudio')
ydl = YDL({})
self.assertEqual(ydl._default_format_spec({}, download=False), 'bestvideo+bestaudio/best')
self.assertEqual(ydl._default_format_spec({'is_live': True}), 'best/bestvideo+bestaudio')
class TestYoutubeDL(unittest.TestCase):
def test_subtitles(self):
@ -525,6 +627,9 @@ class TestYoutubeDL(unittest.TestCase):
'id': '1234',
'ext': 'mp4',
'width': None,
'height': 1080,
'title1': '$PATH',
'title2': '%PATH%',
}
def fname(templ):
@ -534,16 +639,33 @@ class TestYoutubeDL(unittest.TestCase):
self.assertEqual(fname('%(id)s-%(width)s.%(ext)s'), '1234-NA.mp4')
# Replace missing fields with 'NA'
self.assertEqual(fname('%(uploader_date)s-%(id)s.%(ext)s'), 'NA-1234.mp4')
self.assertEqual(fname('%(height)d.%(ext)s'), '1080.mp4')
self.assertEqual(fname('%(height)6d.%(ext)s'), ' 1080.mp4')
self.assertEqual(fname('%(height)-6d.%(ext)s'), '1080 .mp4')
self.assertEqual(fname('%(height)06d.%(ext)s'), '001080.mp4')
self.assertEqual(fname('%(height) 06d.%(ext)s'), ' 01080.mp4')
self.assertEqual(fname('%(height) 06d.%(ext)s'), ' 01080.mp4')
self.assertEqual(fname('%(height)0 6d.%(ext)s'), ' 01080.mp4')
self.assertEqual(fname('%(height)0 6d.%(ext)s'), ' 01080.mp4')
self.assertEqual(fname('%(height) 0 6d.%(ext)s'), ' 01080.mp4')
self.assertEqual(fname('%%'), '%')
self.assertEqual(fname('%%%%'), '%%')
self.assertEqual(fname('%%(height)06d.%(ext)s'), '%(height)06d.mp4')
self.assertEqual(fname('%(width)06d.%(ext)s'), 'NA.mp4')
self.assertEqual(fname('%(width)06d.%%(ext)s'), 'NA.%(ext)s')
self.assertEqual(fname('%%(width)06d.%(ext)s'), '%(width)06d.mp4')
self.assertEqual(fname('Hello %(title1)s'), 'Hello $PATH')
self.assertEqual(fname('Hello %(title2)s'), 'Hello %PATH%')
def test_format_note(self):
ydl = YoutubeDL()
self.assertEqual(ydl._format_note({}), '')
assertRegexpMatches(self, ydl._format_note({
'vbr': 10,
}), '^\s*10k$')
}), r'^\s*10k$')
assertRegexpMatches(self, ydl._format_note({
'fps': 30,
}), '^30fps$')
}), r'^30fps$')
def test_postprocessors(self):
filename = 'post-processor-testfile.mp4'
@ -606,6 +728,8 @@ class TestYoutubeDL(unittest.TestCase):
'duration': 30,
'filesize': 10 * 1024,
'playlist_id': '42',
'uploader': "變態妍字幕版 太妍 тест",
'creator': "тест ' 123 ' тест--",
}
second = {
'id': '2',
@ -616,6 +740,7 @@ class TestYoutubeDL(unittest.TestCase):
'description': 'foo',
'filesize': 5 * 1024,
'playlist_id': '43',
'uploader': "тест 123",
}
videos = [first, second]
@ -656,6 +781,26 @@ class TestYoutubeDL(unittest.TestCase):
res = get_videos(f)
self.assertEqual(res, ['1'])
f = match_filter_func('uploader = "變態妍字幕版 太妍 тест"')
res = get_videos(f)
self.assertEqual(res, ['1'])
f = match_filter_func('uploader != "變態妍字幕版 太妍 тест"')
res = get_videos(f)
self.assertEqual(res, ['2'])
f = match_filter_func('creator = "тест \' 123 \' тест--"')
res = get_videos(f)
self.assertEqual(res, ['1'])
f = match_filter_func("creator = 'тест \\' 123 \\' тест--'")
res = get_videos(f)
self.assertEqual(res, ['1'])
f = match_filter_func(r"creator = 'тест \' 123 \' тест--' & duration > 30")
res = get_videos(f)
self.assertEqual(res, [])
def test_playlist_items_selection(self):
entries = [{
'id': compat_str(i),
@ -671,11 +816,15 @@ class TestYoutubeDL(unittest.TestCase):
'webpage_url': 'http://example.com',
}
def get_ids(params):
def get_downloaded_info_dicts(params):
ydl = YDL(params)
# make a copy because the dictionary can be modified
ydl.process_ie_result(playlist.copy())
return [int(v['id']) for v in ydl.downloaded_info_dicts]
# make a deep copy because the dictionary and nested entries
# can be modified
ydl.process_ie_result(copy.deepcopy(playlist))
return ydl.downloaded_info_dicts
def get_ids(params):
return [int(v['id']) for v in get_downloaded_info_dicts(params)]
result = get_ids({})
self.assertEqual(result, [1, 2, 3, 4])
@ -701,8 +850,30 @@ class TestYoutubeDL(unittest.TestCase):
result = get_ids({'playlist_items': '10'})
self.assertEqual(result, [])
result = get_ids({'playlist_items': '3-10'})
self.assertEqual(result, [3, 4])
result = get_ids({'playlist_items': '2-4,3-4,3'})
self.assertEqual(result, [2, 3, 4])
# Tests for https://github.com/ytdl-org/youtube-dl/issues/10591
# @{
result = get_downloaded_info_dicts({'playlist_items': '2-4,3-4,3'})
self.assertEqual(result[0]['playlist_index'], 2)
self.assertEqual(result[1]['playlist_index'], 3)
result = get_downloaded_info_dicts({'playlist_items': '2-4,3-4,3'})
self.assertEqual(result[0]['playlist_index'], 2)
self.assertEqual(result[1]['playlist_index'], 3)
self.assertEqual(result[2]['playlist_index'], 4)
result = get_downloaded_info_dicts({'playlist_items': '4,2'})
self.assertEqual(result[0]['playlist_index'], 4)
self.assertEqual(result[1]['playlist_index'], 2)
# @}
def test_urlopen_no_file_protocol(self):
# see https://github.com/rg3/youtube-dl/issues/8227
# see https://github.com/ytdl-org/youtube-dl/issues/8227
ydl = YDL()
self.assertRaises(compat_urllib_error.URLError, ydl.urlopen, 'file:///etc/passwd')
@ -717,6 +888,8 @@ class TestYoutubeDL(unittest.TestCase):
'_type': 'url_transparent',
'url': 'foo2:',
'ie_key': 'Foo2',
'title': 'foo1 title',
'id': 'foo1_id',
}
class Foo2IE(InfoExtractor):
@ -733,7 +906,7 @@ class TestYoutubeDL(unittest.TestCase):
_VALID_URL = r'foo3:'
def _real_extract(self, url):
return _make_result([{'url': TEST_URL}])
return _make_result([{'url': TEST_URL}], title='foo3 title')
ydl.add_info_extractor(Foo1IE(ydl))
ydl.add_info_extractor(Foo2IE(ydl))
@ -741,6 +914,80 @@ class TestYoutubeDL(unittest.TestCase):
ydl.extract_info('foo1:')
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['url'], TEST_URL)
self.assertEqual(downloaded['title'], 'foo1 title')
self.assertEqual(downloaded['id'], 'testid')
self.assertEqual(downloaded['extractor'], 'testex')
self.assertEqual(downloaded['extractor_key'], 'TestEx')
# Test case for https://github.com/ytdl-org/youtube-dl/issues/27064
def test_ignoreerrors_for_playlist_with_url_transparent_iterable_entries(self):
class _YDL(YDL):
def __init__(self, *args, **kwargs):
super(_YDL, self).__init__(*args, **kwargs)
def trouble(self, s, tb=None):
pass
ydl = _YDL({
'format': 'extra',
'ignoreerrors': True,
})
class VideoIE(InfoExtractor):
_VALID_URL = r'video:(?P<id>\d+)'
def _real_extract(self, url):
video_id = self._match_id(url)
formats = [{
'format_id': 'default',
'url': 'url:',
}]
if video_id == '0':
raise ExtractorError('foo')
if video_id == '2':
formats.append({
'format_id': 'extra',
'url': TEST_URL,
})
return {
'id': video_id,
'title': 'Video %s' % video_id,
'formats': formats,
}
class PlaylistIE(InfoExtractor):
_VALID_URL = r'playlist:'
def _entries(self):
for n in range(3):
video_id = compat_str(n)
yield {
'_type': 'url_transparent',
'ie_key': VideoIE.ie_key(),
'id': video_id,
'url': 'video:%s' % video_id,
'title': 'Video Transparent %s' % video_id,
}
def _real_extract(self, url):
return self.playlist_result(self._entries())
ydl.add_info_extractor(VideoIE(ydl))
ydl.add_info_extractor(PlaylistIE(ydl))
info = ydl.extract_info('playlist:')
entries = info['entries']
self.assertEqual(len(entries), 3)
self.assertTrue(entries[0] is None)
self.assertTrue(entries[1] is None)
self.assertEqual(len(ydl.downloaded_info_dicts), 1)
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(entries[2], downloaded)
self.assertEqual(downloaded['url'], TEST_URL)
self.assertEqual(downloaded['title'], 'Video Transparent 2')
self.assertEqual(downloaded['id'], '2')
self.assertEqual(downloaded['extractor'], 'Video')
self.assertEqual(downloaded['extractor_key'], 'Video')
if __name__ == '__main__':

View File

@ -0,0 +1,51 @@
#!/usr/bin/env python
# coding: utf-8
from __future__ import unicode_literals
import os
import re
import sys
import tempfile
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from youtube_dl.utils import YoutubeDLCookieJar
class TestYoutubeDLCookieJar(unittest.TestCase):
def test_keep_session_cookies(self):
cookiejar = YoutubeDLCookieJar('./test/testdata/cookies/session_cookies.txt')
cookiejar.load(ignore_discard=True, ignore_expires=True)
tf = tempfile.NamedTemporaryFile(delete=False)
try:
cookiejar.save(filename=tf.name, ignore_discard=True, ignore_expires=True)
temp = tf.read().decode('utf-8')
self.assertTrue(re.search(
r'www\.foobar\.foobar\s+FALSE\s+/\s+TRUE\s+0\s+YoutubeDLExpiresEmpty\s+YoutubeDLExpiresEmptyValue', temp))
self.assertTrue(re.search(
r'www\.foobar\.foobar\s+FALSE\s+/\s+TRUE\s+0\s+YoutubeDLExpires0\s+YoutubeDLExpires0Value', temp))
finally:
tf.close()
os.remove(tf.name)
def test_strip_httponly_prefix(self):
cookiejar = YoutubeDLCookieJar('./test/testdata/cookies/httponly_cookies.txt')
cookiejar.load(ignore_discard=True, ignore_expires=True)
def assert_cookie_has_value(key):
self.assertEqual(cookiejar._cookies['www.foobar.foobar']['/'][key].value, key + '_VALUE')
assert_cookie_has_value('HTTPONLY_COOKIE')
assert_cookie_has_value('JS_ACCESSIBLE_COOKIE')
def test_malformed_cookies(self):
cookiejar = YoutubeDLCookieJar('./test/testdata/cookies/malformed_cookies.txt')
cookiejar.load(ignore_discard=True, ignore_expires=True)
# Cookies should be empty since all malformed cookie file entries
# will be ignored
self.assertFalse(cookiejar._cookies)
if __name__ == '__main__':
unittest.main()

View File

@ -8,7 +8,7 @@ import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from youtube_dl.aes import aes_decrypt, aes_encrypt, aes_cbc_decrypt, aes_decrypt_text
from youtube_dl.aes import aes_decrypt, aes_encrypt, aes_cbc_decrypt, aes_cbc_encrypt, aes_decrypt_text
from youtube_dl.utils import bytes_to_intlist, intlist_to_bytes
import base64
@ -34,19 +34,26 @@ class TestAES(unittest.TestCase):
decrypted = intlist_to_bytes(aes_cbc_decrypt(data, self.key, self.iv))
self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
def test_cbc_encrypt(self):
data = bytes_to_intlist(self.secret_msg)
encrypted = intlist_to_bytes(aes_cbc_encrypt(data, self.key, self.iv))
self.assertEqual(
encrypted,
b"\x97\x92+\xe5\x0b\xc3\x18\x91ky9m&\xb3\xb5@\xe6'\xc2\x96.\xc8u\x88\xab9-[\x9e|\xf1\xcd")
def test_decrypt_text(self):
password = intlist_to_bytes(self.key).decode('utf-8')
encrypted = base64.b64encode(
intlist_to_bytes(self.iv[:8]) +
b'\x17\x15\x93\xab\x8d\x80V\xcdV\xe0\t\xcdo\xc2\xa5\xd8ksM\r\xe27N\xae'
intlist_to_bytes(self.iv[:8])
+ b'\x17\x15\x93\xab\x8d\x80V\xcdV\xe0\t\xcdo\xc2\xa5\xd8ksM\r\xe27N\xae'
).decode('utf-8')
decrypted = (aes_decrypt_text(encrypted, password, 16))
self.assertEqual(decrypted, self.secret_msg)
password = intlist_to_bytes(self.key).decode('utf-8')
encrypted = base64.b64encode(
intlist_to_bytes(self.iv[:8]) +
b'\x0b\xe6\xa4\xd9z\x0e\xb8\xb9\xd0\xd4i_\x85\x1d\x99\x98_\xe5\x80\xe7.\xbf\xa5\x83'
intlist_to_bytes(self.iv[:8])
+ b'\x0b\xe6\xa4\xd9z\x0e\xb8\xb9\xd0\xd4i_\x85\x1d\x99\x98_\xe5\x80\xe7.\xbf\xa5\x83'
).decode('utf-8')
decrypted = (aes_decrypt_text(encrypted, password, 32))
self.assertEqual(decrypted, self.secret_msg)

View File

@ -31,16 +31,17 @@ class TestAllURLsMatching(unittest.TestCase):
def test_youtube_playlist_matching(self):
assertPlaylist = lambda url: self.assertMatch(url, ['youtube:playlist'])
assertTab = lambda url: self.assertMatch(url, ['youtube:tab'])
assertPlaylist('ECUl4u3cNGP61MdtwGTqZA0MreSaDybji8')
assertPlaylist('UUBABnxM4Ar9ten8Mdjj1j0Q') # 585
assertPlaylist('PL63F0C78739B09958')
assertPlaylist('https://www.youtube.com/playlist?list=UUBABnxM4Ar9ten8Mdjj1j0Q')
assertTab('https://www.youtube.com/playlist?list=UUBABnxM4Ar9ten8Mdjj1j0Q')
assertPlaylist('https://www.youtube.com/course?list=ECUl4u3cNGP61MdtwGTqZA0MreSaDybji8')
assertPlaylist('https://www.youtube.com/playlist?list=PLwP_SiAcdui0KVebT0mU9Apz359a4ubsC')
assertPlaylist('https://www.youtube.com/watch?v=AV6J6_AeFEQ&playnext=1&list=PL4023E734DA416012') # 668
assertTab('https://www.youtube.com/playlist?list=PLwP_SiAcdui0KVebT0mU9Apz359a4ubsC')
assertTab('https://www.youtube.com/watch?v=AV6J6_AeFEQ&playnext=1&list=PL4023E734DA416012') # 668
self.assertFalse('youtube:playlist' in self.matching_ies('PLtS2H6bU1M'))
# Top tracks
assertPlaylist('https://www.youtube.com/playlist?list=MCUS.20142101')
assertTab('https://www.youtube.com/playlist?list=MCUS.20142101')
def test_youtube_matching(self):
self.assertTrue(YoutubeIE.suitable('PLtS2H6bU1M'))
@ -51,26 +52,23 @@ class TestAllURLsMatching(unittest.TestCase):
self.assertMatch('http://www.cleanvideosearch.com/media/action/yt/watch?videoId=8v_4O44sfjM', ['youtube'])
def test_youtube_channel_matching(self):
assertChannel = lambda url: self.assertMatch(url, ['youtube:channel'])
assertChannel = lambda url: self.assertMatch(url, ['youtube:tab'])
assertChannel('https://www.youtube.com/channel/HCtnHdj3df7iM')
assertChannel('https://www.youtube.com/channel/HCtnHdj3df7iM?feature=gb_ch_rec')
assertChannel('https://www.youtube.com/channel/HCtnHdj3df7iM/videos')
def test_youtube_user_matching(self):
self.assertMatch('http://www.youtube.com/NASAgovVideo/videos', ['youtube:user'])
# def test_youtube_user_matching(self):
# self.assertMatch('http://www.youtube.com/NASAgovVideo/videos', ['youtube:tab'])
def test_youtube_feeds(self):
self.assertMatch('https://www.youtube.com/feed/watch_later', ['youtube:watchlater'])
self.assertMatch('https://www.youtube.com/feed/subscriptions', ['youtube:subscriptions'])
self.assertMatch('https://www.youtube.com/feed/recommended', ['youtube:recommended'])
self.assertMatch('https://www.youtube.com/my_favorites', ['youtube:favorites'])
self.assertMatch('https://www.youtube.com/feed/library', ['youtube:tab'])
self.assertMatch('https://www.youtube.com/feed/history', ['youtube:tab'])
self.assertMatch('https://www.youtube.com/feed/watch_later', ['youtube:tab'])
self.assertMatch('https://www.youtube.com/feed/subscriptions', ['youtube:tab'])
def test_youtube_show_matching(self):
self.assertMatch('http://www.youtube.com/show/airdisasters', ['youtube:show'])
def test_youtube_search_matching(self):
self.assertMatch('http://www.youtube.com/results?search_query=making+mustard', ['youtube:search_url'])
self.assertMatch('https://www.youtube.com/results?baz=bar&search_query=youtube-dl+test+video&filters=video&lclk=video', ['youtube:search_url'])
# def test_youtube_search_matching(self):
# self.assertMatch('http://www.youtube.com/results?search_query=making+mustard', ['youtube:search_url'])
# self.assertMatch('https://www.youtube.com/results?baz=bar&search_query=youtube-dl+test+video&filters=video&lclk=video', ['youtube:search_url'])
def test_youtube_extract(self):
assertExtractId = lambda url, id: self.assertEqual(YoutubeIE.extract_id(url), id)
@ -110,7 +108,7 @@ class TestAllURLsMatching(unittest.TestCase):
self.assertMatch('https://vimeo.com/user7108434/videos', ['vimeo:user'])
self.assertMatch('https://vimeo.com/user21297594/review/75524534/3c257a1b5d', ['vimeo:review'])
# https://github.com/rg3/youtube-dl/issues/1930
# https://github.com/ytdl-org/youtube-dl/issues/1930
def test_soundcloud_not_matching_sets(self):
self.assertMatch('http://soundcloud.com/floex/sets/gone-ep', ['soundcloud:set'])
@ -119,16 +117,10 @@ class TestAllURLsMatching(unittest.TestCase):
self.assertMatch('http://tatianamaslanydaily.tumblr.com/post/54196191430', ['Tumblr'])
def test_pbs(self):
# https://github.com/rg3/youtube-dl/issues/2350
# https://github.com/ytdl-org/youtube-dl/issues/2350
self.assertMatch('http://video.pbs.org/viralplayer/2365173446/', ['pbs'])
self.assertMatch('http://video.pbs.org/widget/partnerplayer/980042464/', ['pbs'])
def test_yahoo_https(self):
# https://github.com/rg3/youtube-dl/issues/2701
self.assertMatch(
'https://screen.yahoo.com/smartwatches-latest-wearable-gadgets-163745379-cbs.html',
['Yahoo'])
def test_no_duplicated_ie_names(self):
name_accu = collections.defaultdict(list)
for ie in self.ies:

View File

@ -13,6 +13,7 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from youtube_dl.compat import (
compat_getenv,
compat_setenv,
compat_etree_Element,
compat_etree_fromstring,
compat_expanduser,
compat_shlex_split,
@ -27,11 +28,11 @@ from youtube_dl.compat import (
class TestCompat(unittest.TestCase):
def test_compat_getenv(self):
test_str = 'тест'
compat_setenv('YOUTUBE-DL-TEST', test_str)
self.assertEqual(compat_getenv('YOUTUBE-DL-TEST'), test_str)
compat_setenv('YOUTUBE_DL_COMPAT_GETENV', test_str)
self.assertEqual(compat_getenv('YOUTUBE_DL_COMPAT_GETENV'), test_str)
def test_compat_setenv(self):
test_var = 'YOUTUBE-DL-TEST'
test_var = 'YOUTUBE_DL_COMPAT_SETENV'
test_str = 'тест'
compat_setenv(test_var, test_str)
compat_getenv(test_var)
@ -39,7 +40,7 @@ class TestCompat(unittest.TestCase):
def test_compat_expanduser(self):
old_home = os.environ.get('HOME')
test_str = 'C:\Documents and Settings\тест\Application Data'
test_str = r'C:\Documents and Settings\тест\Application Data'
compat_setenv('HOME', test_str)
self.assertEqual(compat_expanduser('~'), test_str)
compat_setenv('HOME', old_home or '')
@ -90,6 +91,12 @@ class TestCompat(unittest.TestCase):
self.assertEqual(compat_shlex_split('-option "one\ntwo" \n -flag'), ['-option', 'one\ntwo', '-flag'])
self.assertEqual(compat_shlex_split('-val 中文'), ['-val', '中文'])
def test_compat_etree_Element(self):
try:
compat_etree_Element.items
except AttributeError:
self.fail('compat_etree_Element is not a type')
def test_compat_etree_fromstring(self):
xml = '''
<root foo="bar" spam="中文">

View File

@ -65,19 +65,35 @@ defs = gettestcases()
class TestDownload(unittest.TestCase):
# Parallel testing in nosetests. See
# http://nose.readthedocs.org/en/latest/doc_tests/test_multiprocess/multiprocess.html
_multiprocess_shared_ = True
maxDiff = None
def __str__(self):
"""Identify each test with the `add_ie` attribute, if available."""
def strclass(cls):
"""From 2.7's unittest; 2.6 had _strclass so we can't import it."""
return '%s.%s' % (cls.__module__, cls.__name__)
add_ie = getattr(self, self._testMethodName).add_ie
return '%s (%s)%s:' % (self._testMethodName,
strclass(self.__class__),
' [%s]' % add_ie if add_ie else '')
def setUp(self):
self.defs = defs
# Dynamically generate tests
def generator(test_case):
def generator(test_case, tname):
def test_template(self):
ie = youtube_dl.extractor.get_info_extractor(test_case['name'])
other_ies = [get_info_extractor(ie_key) for ie_key in test_case.get('add_ie', [])]
ie = youtube_dl.extractor.get_info_extractor(test_case['name'])()
other_ies = [get_info_extractor(ie_key)() for ie_key in test_case.get('add_ie', [])]
is_playlist = any(k.startswith('playlist') for k in test_case)
test_cases = test_case.get(
'playlist', [] if is_playlist else [test_case])
@ -102,6 +118,7 @@ def generator(test_case):
return
params = get_params(test_case.get('params', {}))
params['outtmpl'] = tname + '_' + params['outtmpl']
if is_playlist and 'playlist' not in test_case:
params.setdefault('extract_flat', 'in_playlist')
params.setdefault('skip_download', True)
@ -134,7 +151,7 @@ def generator(test_case):
try_num = 1
while True:
try:
# We're not using .download here sine that is just a shim
# We're not using .download here since that is just a shim
# for outside error handling, and returns the exit code
# instead of the result dict.
res_dict = ydl.extract_info(
@ -146,7 +163,7 @@ def generator(test_case):
raise
if try_num == RETRIES:
report_warning('Failed due to network errors, skipping...')
report_warning('%s failed due to network errors, skipping...' % tname)
return
print('Retrying: {0} failed tries\n\n##########\n\n'.format(try_num))
@ -182,7 +199,16 @@ def generator(test_case):
self.assertEqual(
test_case['playlist_duration_sum'], got_duration)
for tc in test_cases:
# Generalize both playlists and single videos to unified format for
# simplicity
if 'entries' not in res_dict:
res_dict['entries'] = [res_dict]
for tc_num, tc in enumerate(test_cases):
tc_res_dict = res_dict['entries'][tc_num]
# First, check test cases' data against extracted data alone
expect_info_dict(self, tc_res_dict, tc.get('info_dict', {}))
# Now, check downloaded file consistency
tc_filename = get_tc_filename(tc)
if not test_case.get('params', {}).get('skip_download', False):
self.assertTrue(os.path.exists(tc_filename), msg='Missing file ' + tc_filename)
@ -199,14 +225,15 @@ def generator(test_case):
format_bytes(got_fsize)))
if 'md5' in tc:
md5_for_file = _file_md5(tc_filename)
self.assertEqual(md5_for_file, tc['md5'])
self.assertEqual(tc['md5'], md5_for_file)
# Finally, check test cases' data again but this time against
# extracted data from info JSON file written during processing
info_json_fn = os.path.splitext(tc_filename)[0] + '.info.json'
self.assertTrue(
os.path.exists(info_json_fn),
'Missing info file %s' % info_json_fn)
with io.open(info_json_fn, encoding='utf-8') as infof:
info_dict = json.load(infof)
expect_info_dict(self, info_dict, tc.get('info_dict', {}))
finally:
try_rm_tcs_files()
@ -221,13 +248,15 @@ def generator(test_case):
# And add them to TestDownload
for n, test_case in enumerate(defs):
test_method = generator(test_case)
tname = 'test_' + str(test_case['name'])
i = 1
while hasattr(TestDownload, tname):
tname = 'test_%s_%d' % (test_case['name'], i)
i += 1
test_method = generator(test_case, tname)
test_method.__name__ = str(tname)
ie_list = test_case.get('add_ie')
test_method.add_ie = ie_list and ','.join(ie_list)
setattr(TestDownload, test_method.__name__, test_method)
del test_method

View File

@ -0,0 +1,115 @@
#!/usr/bin/env python
# coding: utf-8
from __future__ import unicode_literals
# Allow direct execution
import os
import re
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from test.helper import http_server_port, try_rm
from youtube_dl import YoutubeDL
from youtube_dl.compat import compat_http_server
from youtube_dl.downloader.http import HttpFD
from youtube_dl.utils import encodeFilename
import threading
TEST_DIR = os.path.dirname(os.path.abspath(__file__))
TEST_SIZE = 10 * 1024
class HTTPTestRequestHandler(compat_http_server.BaseHTTPRequestHandler):
def log_message(self, format, *args):
pass
def send_content_range(self, total=None):
range_header = self.headers.get('Range')
start = end = None
if range_header:
mobj = re.search(r'^bytes=(\d+)-(\d+)', range_header)
if mobj:
start = int(mobj.group(1))
end = int(mobj.group(2))
valid_range = start is not None and end is not None
if valid_range:
content_range = 'bytes %d-%d' % (start, end)
if total:
content_range += '/%d' % total
self.send_header('Content-Range', content_range)
return (end - start + 1) if valid_range else total
def serve(self, range=True, content_length=True):
self.send_response(200)
self.send_header('Content-Type', 'video/mp4')
size = TEST_SIZE
if range:
size = self.send_content_range(TEST_SIZE)
if content_length:
self.send_header('Content-Length', size)
self.end_headers()
self.wfile.write(b'#' * size)
def do_GET(self):
if self.path == '/regular':
self.serve()
elif self.path == '/no-content-length':
self.serve(content_length=False)
elif self.path == '/no-range':
self.serve(range=False)
elif self.path == '/no-range-no-content-length':
self.serve(range=False, content_length=False)
else:
assert False
class FakeLogger(object):
def debug(self, msg):
pass
def warning(self, msg):
pass
def error(self, msg):
pass
class TestHttpFD(unittest.TestCase):
def setUp(self):
self.httpd = compat_http_server.HTTPServer(
('127.0.0.1', 0), HTTPTestRequestHandler)
self.port = http_server_port(self.httpd)
self.server_thread = threading.Thread(target=self.httpd.serve_forever)
self.server_thread.daemon = True
self.server_thread.start()
def download(self, params, ep):
params['logger'] = FakeLogger()
ydl = YoutubeDL(params)
downloader = HttpFD(ydl, params)
filename = 'testfile.mp4'
try_rm(encodeFilename(filename))
self.assertTrue(downloader.real_download(filename, {
'url': 'http://127.0.0.1:%d/%s' % (self.port, ep),
}))
self.assertEqual(os.path.getsize(encodeFilename(filename)), TEST_SIZE)
try_rm(encodeFilename(filename))
def download_all(self, params):
for ep in ('regular', 'no-content-length', 'no-range', 'no-range-no-content-length'):
self.download(params, ep)
def test_regular(self):
self.download_all({})
def test_chunked(self):
self.download_all({
'http_chunk_size': 1000,
})
if __name__ == '__main__':
unittest.main()

View File

@ -8,6 +8,7 @@ import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from test.helper import http_server_port
from youtube_dl import YoutubeDL
from youtube_dl.compat import compat_http_server, compat_urllib_request
import ssl
@ -16,15 +17,6 @@ import threading
TEST_DIR = os.path.dirname(os.path.abspath(__file__))
def http_server_port(httpd):
if os.name == 'java' and isinstance(httpd.socket, ssl.SSLSocket):
# In Jython SSLSocket is not a subclass of socket.socket
sock = httpd.socket.sock
else:
sock = httpd.socket
return sock.getsockname()[1]
class HTTPTestRequestHandler(compat_http_server.BaseHTTPRequestHandler):
def log_message(self, format, *args):
pass
@ -47,7 +39,7 @@ class HTTPTestRequestHandler(compat_http_server.BaseHTTPRequestHandler):
self.end_headers()
return
new_url = 'http://localhost:%d/中文.html' % http_server_port(self.server)
new_url = 'http://127.0.0.1:%d/中文.html' % http_server_port(self.server)
self.send_response(302)
self.send_header(b'Location', new_url.encode('utf-8'))
self.end_headers()
@ -74,7 +66,7 @@ class FakeLogger(object):
class TestHTTP(unittest.TestCase):
def setUp(self):
self.httpd = compat_http_server.HTTPServer(
('localhost', 0), HTTPTestRequestHandler)
('127.0.0.1', 0), HTTPTestRequestHandler)
self.port = http_server_port(self.httpd)
self.server_thread = threading.Thread(target=self.httpd.serve_forever)
self.server_thread.daemon = True
@ -86,15 +78,15 @@ class TestHTTP(unittest.TestCase):
return
ydl = YoutubeDL({'logger': FakeLogger()})
r = ydl.extract_info('http://localhost:%d/302' % self.port)
self.assertEqual(r['entries'][0]['url'], 'http://localhost:%d/vid.mp4' % self.port)
r = ydl.extract_info('http://127.0.0.1:%d/302' % self.port)
self.assertEqual(r['entries'][0]['url'], 'http://127.0.0.1:%d/vid.mp4' % self.port)
class TestHTTPS(unittest.TestCase):
def setUp(self):
certfn = os.path.join(TEST_DIR, 'testcert.pem')
self.httpd = compat_http_server.HTTPServer(
('localhost', 0), HTTPTestRequestHandler)
('127.0.0.1', 0), HTTPTestRequestHandler)
self.httpd.socket = ssl.wrap_socket(
self.httpd.socket, certfile=certfn, server_side=True)
self.port = http_server_port(self.httpd)
@ -107,11 +99,11 @@ class TestHTTPS(unittest.TestCase):
ydl = YoutubeDL({'logger': FakeLogger()})
self.assertRaises(
Exception,
ydl.extract_info, 'https://localhost:%d/video.html' % self.port)
ydl.extract_info, 'https://127.0.0.1:%d/video.html' % self.port)
ydl = YoutubeDL({'logger': FakeLogger(), 'nocheckcertificate': True})
r = ydl.extract_info('https://localhost:%d/video.html' % self.port)
self.assertEqual(r['entries'][0]['url'], 'https://localhost:%d/vid.mp4' % self.port)
r = ydl.extract_info('https://127.0.0.1:%d/video.html' % self.port)
self.assertEqual(r['entries'][0]['url'], 'https://127.0.0.1:%d/vid.mp4' % self.port)
def _build_proxy_handler(name):
@ -132,23 +124,23 @@ def _build_proxy_handler(name):
class TestProxy(unittest.TestCase):
def setUp(self):
self.proxy = compat_http_server.HTTPServer(
('localhost', 0), _build_proxy_handler('normal'))
('127.0.0.1', 0), _build_proxy_handler('normal'))
self.port = http_server_port(self.proxy)
self.proxy_thread = threading.Thread(target=self.proxy.serve_forever)
self.proxy_thread.daemon = True
self.proxy_thread.start()
self.geo_proxy = compat_http_server.HTTPServer(
('localhost', 0), _build_proxy_handler('geo'))
('127.0.0.1', 0), _build_proxy_handler('geo'))
self.geo_port = http_server_port(self.geo_proxy)
self.geo_proxy_thread = threading.Thread(target=self.geo_proxy.serve_forever)
self.geo_proxy_thread.daemon = True
self.geo_proxy_thread.start()
def test_proxy(self):
geo_proxy = 'localhost:{0}'.format(self.geo_port)
geo_proxy = '127.0.0.1:{0}'.format(self.geo_port)
ydl = YoutubeDL({
'proxy': 'localhost:{0}'.format(self.port),
'proxy': '127.0.0.1:{0}'.format(self.port),
'geo_verification_proxy': geo_proxy,
})
url = 'http://foo.com/bar'
@ -162,7 +154,7 @@ class TestProxy(unittest.TestCase):
def test_proxy_with_idn(self):
ydl = YoutubeDL({
'proxy': 'localhost:{0}'.format(self.port),
'proxy': '127.0.0.1:{0}'.format(self.port),
})
url = 'http://中文.tw/'
response = ydl.urlopen(url).read().decode('utf-8')

26
test/test_options.py Normal file
View File

@ -0,0 +1,26 @@
# coding: utf-8
from __future__ import unicode_literals
# Allow direct execution
import os
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from youtube_dl.options import _hide_login_info
class TestOptions(unittest.TestCase):
def test_hide_login_info(self):
self.assertEqual(_hide_login_info(['-u', 'foo', '-p', 'bar']),
['-u', 'PRIVATE', '-p', 'PRIVATE'])
self.assertEqual(_hide_login_info(['-u']), ['-u'])
self.assertEqual(_hide_login_info(['-u', 'foo', '-u', 'bar']),
['-u', 'PRIVATE', '-u', 'PRIVATE'])
self.assertEqual(_hide_login_info(['--username=foo']),
['--username=PRIVATE'])
if __name__ == '__main__':
unittest.main()

View File

@ -14,4 +14,4 @@ from youtube_dl.postprocessor import MetadataFromTitlePP
class TestMetadataFromTitle(unittest.TestCase):
def test_format_to_regex(self):
pp = MetadataFromTitlePP(None, '%(title)s - %(artist)s')
self.assertEqual(pp._titleregex, '(?P<title>.+)\ \-\ (?P<artist>.+)')
self.assertEqual(pp._titleregex, r'(?P<title>.+)\ \-\ (?P<artist>.+)')

View File

@ -21,12 +21,11 @@ from youtube_dl.extractor import (
NPOIE,
ComedyCentralIE,
NRKTVIE,
RaiTVIE,
RaiPlayIE,
VikiIE,
ThePlatformIE,
ThePlatformFeedIE,
RTVEALaCartaIE,
FunnyOrDieIE,
DemocracynowIE,
)
@ -232,7 +231,7 @@ class TestNPOSubtitles(BaseTestSubtitles):
class TestMTVSubtitles(BaseTestSubtitles):
url = 'http://www.cc.com/video-clips/kllhuv/stand-up-greg-fitzsimmons--uncensored---too-good-of-a-mother'
url = 'http://www.cc.com/video-clips/p63lk0/adam-devine-s-house-party-chasing-white-swans'
IE = ComedyCentralIE
def getInfoDict(self):
@ -243,7 +242,7 @@ class TestMTVSubtitles(BaseTestSubtitles):
self.DL.params['allsubtitles'] = True
subtitles = self.getSubtitles()
self.assertEqual(set(subtitles.keys()), set(['en']))
self.assertEqual(md5(subtitles['en']), 'b9f6ca22a6acf597ec76f61749765e65')
self.assertEqual(md5(subtitles['en']), '78206b8d8a0cfa9da64dc026eea48961')
class TestNRKSubtitles(BaseTestSubtitles):
@ -258,9 +257,9 @@ class TestNRKSubtitles(BaseTestSubtitles):
self.assertEqual(md5(subtitles['no']), '544fa917d3197fcbee64634559221cc2')
class TestRaiSubtitles(BaseTestSubtitles):
url = 'http://www.rai.tv/dl/RaiTV/programmi/media/ContentItem-cb27157f-9dd0-4aee-b788-b1f67643a391.html'
IE = RaiTVIE
class TestRaiPlaySubtitles(BaseTestSubtitles):
url = 'http://www.raiplay.it/video/2014/04/Report-del-07042014-cb27157f-9dd0-4aee-b788-b1f67643a391.html'
IE = RaiPlayIE
def test_allsubtitles(self):
self.DL.params['writesubtitles'] = True
@ -322,18 +321,6 @@ class TestRtveSubtitles(BaseTestSubtitles):
self.assertEqual(md5(subtitles['es']), '69e70cae2d40574fb7316f31d6eb7fca')
class TestFunnyOrDieSubtitles(BaseTestSubtitles):
url = 'http://www.funnyordie.com/videos/224829ff6d/judd-apatow-will-direct-your-vine'
IE = FunnyOrDieIE
def test_allsubtitles(self):
self.DL.params['writesubtitles'] = True
self.DL.params['allsubtitles'] = True
subtitles = self.getSubtitles()
self.assertEqual(set(subtitles.keys()), set(['en']))
self.assertEqual(md5(subtitles['en']), 'c5593c193eacd353596c11c2d4f9ecc4')
class TestDemocracynowSubtitles(BaseTestSubtitles):
url = 'http://www.democracynow.org/shows/2015/7/3'
IE = DemocracynowIE

View File

@ -34,8 +34,8 @@ def _make_testfunc(testfile):
def test_func(self):
as_file = os.path.join(TEST_DIR, testfile)
swf_file = os.path.join(TEST_DIR, test_id + '.swf')
if ((not os.path.exists(swf_file)) or
os.path.getmtime(swf_file) < os.path.getmtime(as_file)):
if ((not os.path.exists(swf_file))
or os.path.getmtime(swf_file) < os.path.getmtime(as_file)):
# Recompile
try:
subprocess.check_call([

View File

@ -19,6 +19,7 @@ from youtube_dl.utils import (
age_restricted,
args_to_str,
encode_base_n,
caesar,
clean_html,
date_from_str,
DateRange,
@ -33,17 +34,21 @@ from youtube_dl.utils import (
ExtractorError,
find_xpath_attr,
fix_xml_ampersands,
float_or_none,
get_element_by_class,
get_element_by_attribute,
get_elements_by_class,
get_elements_by_attribute,
InAdvancePagedList,
int_or_none,
intlist_to_bytes,
is_html,
js_to_json,
limit_length,
merge_dicts,
mimetype2ext,
month_by_name,
multipart_encode,
ohdave_rsa_encrypt,
OnDemandPagedList,
orderedSet,
@ -52,18 +57,26 @@ from youtube_dl.utils import (
parse_filesize,
parse_count,
parse_iso8601,
parse_resolution,
parse_bitrate,
pkcs1pad,
read_batch_urls,
sanitize_filename,
sanitize_path,
sanitize_url,
expand_path,
prepend_extension,
replace_extension,
remove_start,
remove_end,
remove_quotes,
rot47,
shell_quote,
smuggle_url,
str_to_int,
strip_jsonp,
strip_or_none,
subtitles_filename,
timeconvert,
unescapeHTML,
unified_strdate,
@ -72,6 +85,7 @@ from youtube_dl.utils import (
uppercase_escape,
lowercase_escape,
url_basename,
url_or_none,
base_url,
urljoin,
urlencode_postdata,
@ -94,6 +108,9 @@ from youtube_dl.utils import (
from youtube_dl.compat import (
compat_chr,
compat_etree_fromstring,
compat_getenv,
compat_os_name,
compat_setenv,
compat_urlparse,
compat_parse_qs,
)
@ -170,7 +187,7 @@ class TestUtil(unittest.TestCase):
self.assertEqual(sanitize_filename(
'ÂÃÄÀÁÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖŐØŒÙÚÛÜŰÝÞßàáâãäåæçèéêëìíîïðñòóôõöőøœùúûüűýþÿ', restricted=True),
'AAAAAAAECEEEEIIIIDNOOOOOOOOEUUUUUYPssaaaaaaaeceeeeiiiionooooooooeuuuuuypy')
'AAAAAAAECEEEEIIIIDNOOOOOOOOEUUUUUYTHssaaaaaaaeceeeeiiiionooooooooeuuuuuythy')
def test_sanitize_ids(self):
self.assertEqual(sanitize_filename('_n_cd26wFpw', is_id=True), '_n_cd26wFpw')
@ -213,6 +230,24 @@ class TestUtil(unittest.TestCase):
self.assertEqual(sanitize_path('./abc'), 'abc')
self.assertEqual(sanitize_path('./../abc'), '..\\abc')
def test_sanitize_url(self):
self.assertEqual(sanitize_url('//foo.bar'), 'http://foo.bar')
self.assertEqual(sanitize_url('httpss://foo.bar'), 'https://foo.bar')
self.assertEqual(sanitize_url('rmtps://foo.bar'), 'rtmps://foo.bar')
self.assertEqual(sanitize_url('https://foo.bar'), 'https://foo.bar')
def test_expand_path(self):
def env(var):
return '%{0}%'.format(var) if sys.platform == 'win32' else '${0}'.format(var)
compat_setenv('YOUTUBE_DL_EXPATH_PATH', 'expanded')
self.assertEqual(expand_path(env('YOUTUBE_DL_EXPATH_PATH')), 'expanded')
self.assertEqual(expand_path(env('HOME')), compat_getenv('HOME'))
self.assertEqual(expand_path('~'), compat_getenv('HOME'))
self.assertEqual(
expand_path('~/%s' % env('YOUTUBE_DL_EXPATH_PATH')),
'%s/expanded' % compat_getenv('HOME'))
def test_prepend_extension(self):
self.assertEqual(prepend_extension('abc.ext', 'temp'), 'abc.temp.ext')
self.assertEqual(prepend_extension('abc.ext', 'temp', 'ext'), 'abc.temp.ext')
@ -229,6 +264,11 @@ class TestUtil(unittest.TestCase):
self.assertEqual(replace_extension('.abc', 'temp'), '.abc.temp')
self.assertEqual(replace_extension('.abc.ext', 'temp'), '.abc.temp')
def test_subtitles_filename(self):
self.assertEqual(subtitles_filename('abc.ext', 'en', 'vtt'), 'abc.en.vtt')
self.assertEqual(subtitles_filename('abc.ext', 'en', 'vtt', 'ext'), 'abc.en.vtt')
self.assertEqual(subtitles_filename('abc.unexpected_ext', 'en', 'vtt', 'ext'), 'abc.unexpected_ext.en.vtt')
def test_remove_start(self):
self.assertEqual(remove_start(None, 'A - '), None)
self.assertEqual(remove_start('A - B', 'A - '), 'B')
@ -261,6 +301,7 @@ class TestUtil(unittest.TestCase):
self.assertEqual(unescapeHTML('&#47;'), '/')
self.assertEqual(unescapeHTML('&eacute;'), 'é')
self.assertEqual(unescapeHTML('&#2013266066;'), '&#2013266066;')
self.assertEqual(unescapeHTML('&a&quot;'), '&a"')
# HTML5 entities
self.assertEqual(unescapeHTML('&period;&apos;'), '.\'')
@ -301,6 +342,8 @@ class TestUtil(unittest.TestCase):
self.assertEqual(unified_strdate('July 15th, 2013'), '20130715')
self.assertEqual(unified_strdate('September 1st, 2013'), '20130901')
self.assertEqual(unified_strdate('Sep 2nd, 2013'), '20130902')
self.assertEqual(unified_strdate('November 3rd, 2019'), '20191103')
self.assertEqual(unified_strdate('October 23rd, 2005'), '20051023')
def test_unified_timestamps(self):
self.assertEqual(unified_timestamp('December 21, 2010'), 1292889600)
@ -322,6 +365,10 @@ class TestUtil(unittest.TestCase):
self.assertEqual(unified_timestamp('UNKNOWN DATE FORMAT'), None)
self.assertEqual(unified_timestamp('May 16, 2016 11:15 PM'), 1463440500)
self.assertEqual(unified_timestamp('Feb 7, 2016 at 6:35 pm'), 1454870100)
self.assertEqual(unified_timestamp('2017-03-30T17:52:41Q'), 1490896361)
self.assertEqual(unified_timestamp('Sep 11, 2013 | 5:49 AM'), 1378878540)
self.assertEqual(unified_timestamp('December 15, 2017 at 7:49 am'), 1513324140)
self.assertEqual(unified_timestamp('2018-03-14T08:32:43.1493874+00:00'), 1521016363)
def test_determine_ext(self):
self.assertEqual(determine_ext('http://example.com/foo/bar.mp4/?download'), 'mp4')
@ -329,6 +376,7 @@ class TestUtil(unittest.TestCase):
self.assertEqual(determine_ext('http://example.com/foo/bar.nonext/?download', None), None)
self.assertEqual(determine_ext('http://example.com/foo/bar/mp4?download', None), None)
self.assertEqual(determine_ext('http://example.com/foo/bar.m3u8//?download'), 'm3u8')
self.assertEqual(determine_ext('foobar', None), None)
def test_find_xpath_attr(self):
testxml = '''<root>
@ -429,11 +477,34 @@ class TestUtil(unittest.TestCase):
def test_shell_quote(self):
args = ['ffmpeg', '-i', encodeFilename('ñ€ß\'.mp4')]
self.assertEqual(shell_quote(args), """ffmpeg -i 'ñ€ß'"'"'.mp4'""")
self.assertEqual(
shell_quote(args),
"""ffmpeg -i 'ñ€ß'"'"'.mp4'""" if compat_os_name != 'nt' else '''ffmpeg -i "ñ€ß'.mp4"''')
def test_float_or_none(self):
self.assertEqual(float_or_none('42.42'), 42.42)
self.assertEqual(float_or_none('42'), 42.0)
self.assertEqual(float_or_none(''), None)
self.assertEqual(float_or_none(None), None)
self.assertEqual(float_or_none([]), None)
self.assertEqual(float_or_none(set()), None)
def test_int_or_none(self):
self.assertEqual(int_or_none('42'), 42)
self.assertEqual(int_or_none(''), None)
self.assertEqual(int_or_none(None), None)
self.assertEqual(int_or_none([]), None)
self.assertEqual(int_or_none(set()), None)
def test_str_to_int(self):
self.assertEqual(str_to_int('123,456'), 123456)
self.assertEqual(str_to_int('123.456'), 123456)
self.assertEqual(str_to_int(523), 523)
# Python 3 has no long
if sys.version_info < (3, 0):
eval('self.assertEqual(str_to_int(123456L), 123456)')
self.assertEqual(str_to_int('noninteger'), None)
self.assertEqual(str_to_int([]), None)
def test_url_basename(self):
self.assertEqual(url_basename('http://foo.de/'), '')
@ -454,6 +525,9 @@ class TestUtil(unittest.TestCase):
def test_urljoin(self):
self.assertEqual(urljoin('http://foo.de/', '/a/b/c.txt'), 'http://foo.de/a/b/c.txt')
self.assertEqual(urljoin(b'http://foo.de/', '/a/b/c.txt'), 'http://foo.de/a/b/c.txt')
self.assertEqual(urljoin('http://foo.de/', b'/a/b/c.txt'), 'http://foo.de/a/b/c.txt')
self.assertEqual(urljoin(b'http://foo.de/', b'/a/b/c.txt'), 'http://foo.de/a/b/c.txt')
self.assertEqual(urljoin('//foo.de/', '/a/b/c.txt'), '//foo.de/a/b/c.txt')
self.assertEqual(urljoin('http://foo.de/', 'a/b/c.txt'), 'http://foo.de/a/b/c.txt')
self.assertEqual(urljoin('http://foo.de', '/a/b/c.txt'), 'http://foo.de/a/b/c.txt')
@ -468,6 +542,18 @@ class TestUtil(unittest.TestCase):
self.assertEqual(urljoin('http://foo.de/', ''), None)
self.assertEqual(urljoin('http://foo.de/', ['foobar']), None)
self.assertEqual(urljoin('http://foo.de/a/b/c.txt', '.././../d.txt'), 'http://foo.de/d.txt')
self.assertEqual(urljoin('http://foo.de/a/b/c.txt', 'rtmp://foo.de'), 'rtmp://foo.de')
self.assertEqual(urljoin(None, 'rtmp://foo.de'), 'rtmp://foo.de')
def test_url_or_none(self):
self.assertEqual(url_or_none(None), None)
self.assertEqual(url_or_none(''), None)
self.assertEqual(url_or_none('foo'), None)
self.assertEqual(url_or_none('http://foo.de'), 'http://foo.de')
self.assertEqual(url_or_none('https://foo.de'), 'https://foo.de')
self.assertEqual(url_or_none('http$://foo.de'), None)
self.assertEqual(url_or_none('http://foo.de'), 'http://foo.de')
self.assertEqual(url_or_none('//foo.de'), '//foo.de')
def test_parse_age_limit(self):
self.assertEqual(parse_age_limit(None), None)
@ -482,6 +568,8 @@ class TestUtil(unittest.TestCase):
self.assertEqual(parse_age_limit('PG-13'), 13)
self.assertEqual(parse_age_limit('TV-14'), 14)
self.assertEqual(parse_age_limit('TV-MA'), 17)
self.assertEqual(parse_age_limit('TV14'), 14)
self.assertEqual(parse_age_limit('TV_G'), 0)
def test_parse_duration(self):
self.assertEqual(parse_duration(None), None)
@ -514,6 +602,7 @@ class TestUtil(unittest.TestCase):
self.assertEqual(parse_duration('87 Min.'), 5220)
self.assertEqual(parse_duration('PT1H0.040S'), 3600.04)
self.assertEqual(parse_duration('PT00H03M30SZ'), 210)
self.assertEqual(parse_duration('P0Y0M0DT0H4M20.880S'), 260.88)
def test_fix_xml_ampersands(self):
self.assertEqual(
@ -600,6 +689,16 @@ class TestUtil(unittest.TestCase):
'http://example.com/path', {'test': '第二行тест'})),
query_dict('http://example.com/path?test=%E7%AC%AC%E4%BA%8C%E8%A1%8C%D1%82%D0%B5%D1%81%D1%82'))
def test_multipart_encode(self):
self.assertEqual(
multipart_encode({b'field': b'value'}, boundary='AAAAAA')[0],
b'--AAAAAA\r\nContent-Disposition: form-data; name="field"\r\n\r\nvalue\r\n--AAAAAA--\r\n')
self.assertEqual(
multipart_encode({'欄位'.encode('utf-8'): ''.encode('utf-8')}, boundary='AAAAAA')[0],
b'--AAAAAA\r\nContent-Disposition: form-data; name="\xe6\xac\x84\xe4\xbd\x8d"\r\n\r\n\xe5\x80\xbc\r\n--AAAAAA--\r\n')
self.assertRaises(
ValueError, multipart_encode, {b'field': b'value'}, boundary='value')
def test_dict_get(self):
FALSE_VALUES = {
'none': None,
@ -622,6 +721,17 @@ class TestUtil(unittest.TestCase):
self.assertEqual(dict_get(d, ('b', 'c', key, )), None)
self.assertEqual(dict_get(d, ('b', 'c', key, ), skip_false_values=False), false_value)
def test_merge_dicts(self):
self.assertEqual(merge_dicts({'a': 1}, {'b': 2}), {'a': 1, 'b': 2})
self.assertEqual(merge_dicts({'a': 1}, {'a': 2}), {'a': 1})
self.assertEqual(merge_dicts({'a': 1}, {'a': None}), {'a': 1})
self.assertEqual(merge_dicts({'a': 1}, {'a': ''}), {'a': 1})
self.assertEqual(merge_dicts({'a': 1}, {}), {'a': 1})
self.assertEqual(merge_dicts({'a': None}, {'a': 1}), {'a': 1})
self.assertEqual(merge_dicts({'a': ''}, {'a': 1}), {'a': ''})
self.assertEqual(merge_dicts({'a': ''}, {'a': 'abc'}), {'a': 'abc'})
self.assertEqual(merge_dicts({'a': None}, {'a': ''}, {'a': 'abc'}), {'a': 'abc'})
def test_encode_compat_str(self):
self.assertEqual(encode_compat_str(b'\xd1\x82\xd0\xb5\xd1\x81\xd1\x82', 'utf-8'), 'тест')
self.assertEqual(encode_compat_str('тест', 'utf-8'), 'тест')
@ -647,6 +757,30 @@ class TestUtil(unittest.TestCase):
d = json.loads(stripped)
self.assertEqual(d, {'status': 'success'})
stripped = strip_jsonp('window.cb && window.cb({"status": "success"});')
d = json.loads(stripped)
self.assertEqual(d, {'status': 'success'})
stripped = strip_jsonp('window.cb && cb({"status": "success"});')
d = json.loads(stripped)
self.assertEqual(d, {'status': 'success'})
stripped = strip_jsonp('({"status": "success"});')
d = json.loads(stripped)
self.assertEqual(d, {'status': 'success'})
def test_strip_or_none(self):
self.assertEqual(strip_or_none(' abc'), 'abc')
self.assertEqual(strip_or_none('abc '), 'abc')
self.assertEqual(strip_or_none(' abc '), 'abc')
self.assertEqual(strip_or_none('\tabc\t'), 'abc')
self.assertEqual(strip_or_none('\n\tabc\n\t'), 'abc')
self.assertEqual(strip_or_none('abc'), 'abc')
self.assertEqual(strip_or_none(''), '')
self.assertEqual(strip_or_none(None), None)
self.assertEqual(strip_or_none(42), None)
self.assertEqual(strip_or_none([]), None)
def test_uppercase_escape(self):
self.assertEqual(uppercase_escape(''), '')
self.assertEqual(uppercase_escape('\\U0001d550'), '𝕐')
@ -669,6 +803,8 @@ class TestUtil(unittest.TestCase):
self.assertEqual(mimetype2ext('text/vtt'), 'vtt')
self.assertEqual(mimetype2ext('text/vtt;charset=utf-8'), 'vtt')
self.assertEqual(mimetype2ext('text/html; charset=utf-8'), 'html')
self.assertEqual(mimetype2ext('audio/x-wav'), 'wav')
self.assertEqual(mimetype2ext('audio/x-wav;codec=pcm'), 'wav')
def test_month_by_name(self):
self.assertEqual(month_by_name(None), None)
@ -700,6 +836,19 @@ class TestUtil(unittest.TestCase):
'vcodec': 'h264',
'acodec': 'aac',
})
self.assertEqual(parse_codecs('av01.0.05M.08'), {
'vcodec': 'av01.0.05M.08',
'acodec': 'none',
})
self.assertEqual(parse_codecs('theora, vorbis'), {
'vcodec': 'theora',
'acodec': 'vorbis',
})
self.assertEqual(parse_codecs('unknownvcodec, unknownacodec'), {
'vcodec': 'unknownvcodec',
'acodec': 'unknownacodec',
})
self.assertEqual(parse_codecs('unknown'), {})
def test_escape_rfc3986(self):
reserved = "!*'();:@&=+$,/?#[]"
@ -768,6 +917,9 @@ class TestUtil(unittest.TestCase):
inp = '''{"duration": "00:01:07"}'''
self.assertEqual(js_to_json(inp), '''{"duration": "00:01:07"}''')
inp = '''{segments: [{"offset":-3.885780586188048e-16,"duration":39.75000000000001}]}'''
self.assertEqual(js_to_json(inp), '''{"segments": [{"offset":-3.885780586188048e-16,"duration":39.75000000000001}]}''')
def test_js_to_json_edgecases(self):
on = js_to_json("{abc_def:'1\\'\\\\2\\\\\\'3\"4'}")
self.assertEqual(json.loads(on), {"abc_def": "1'\\2\\'3\"4"})
@ -785,6 +937,28 @@ class TestUtil(unittest.TestCase):
self.assertEqual(d['x'], 1)
self.assertEqual(d['y'], 'a')
# Just drop ! prefix for now though this results in a wrong value
on = js_to_json('''{
a: !0,
b: !1,
c: !!0,
d: !!42.42,
e: !!![],
f: !"abc",
g: !"",
!42: 42
}''')
self.assertEqual(json.loads(on), {
'a': 0,
'b': 1,
'c': 0,
'd': 42.42,
'e': [],
'f': "abc",
'g': "",
'42': 42
})
on = js_to_json('["abc", "def",]')
self.assertEqual(json.loads(on), ['abc', 'def'])
@ -839,6 +1013,19 @@ class TestUtil(unittest.TestCase):
on = js_to_json('{/*comment\n*/42/*comment\n*/:/*comment\n*/42/*comment\n*/}')
self.assertEqual(json.loads(on), {'42': 42})
on = js_to_json('{42:4.2e1}')
self.assertEqual(json.loads(on), {'42': 42.0})
on = js_to_json('{ "0x40": "0x40" }')
self.assertEqual(json.loads(on), {'0x40': '0x40'})
on = js_to_json('{ "040": "040" }')
self.assertEqual(json.loads(on), {'040': '040'})
def test_js_to_json_malformed(self):
self.assertEqual(js_to_json('42a1'), '42"a1"')
self.assertEqual(js_to_json('42a-1'), '42"a"-1')
def test_extract_attributes(self):
self.assertEqual(extract_attributes('<e x="y">'), {'x': 'y'})
self.assertEqual(extract_attributes("<e x='y'>"), {'x': 'y'})
@ -876,10 +1063,13 @@ class TestUtil(unittest.TestCase):
supports_outside_bmp = False
if supports_outside_bmp:
self.assertEqual(extract_attributes('<e x="Smile &#128512;!">'), {'x': 'Smile \U0001f600!'})
# Malformed HTML should not break attributes extraction on older Python
self.assertEqual(extract_attributes('<mal"formed/>'), {})
def test_clean_html(self):
self.assertEqual(clean_html('a:\nb'), 'a: b')
self.assertEqual(clean_html('a:\n "b"'), 'a: "b"')
self.assertEqual(clean_html('a<br>\xa0b'), 'a\nb')
def test_intlist_to_bytes(self):
self.assertEqual(
@ -889,7 +1079,7 @@ class TestUtil(unittest.TestCase):
def test_args_to_str(self):
self.assertEqual(
args_to_str(['foo', 'ba/r', '-baz', '2 be', '']),
'foo ba/r -baz \'2 be\' \'\''
'foo ba/r -baz \'2 be\' \'\'' if compat_os_name != 'nt' else 'foo ba/r -baz "2 be" ""'
)
def test_parse_filesize(self):
@ -916,6 +1106,23 @@ class TestUtil(unittest.TestCase):
self.assertEqual(parse_count('1.1kk '), 1100000)
self.assertEqual(parse_count('1.1kk views'), 1100000)
def test_parse_resolution(self):
self.assertEqual(parse_resolution(None), {})
self.assertEqual(parse_resolution(''), {})
self.assertEqual(parse_resolution('1920x1080'), {'width': 1920, 'height': 1080})
self.assertEqual(parse_resolution('1920×1080'), {'width': 1920, 'height': 1080})
self.assertEqual(parse_resolution('1920 x 1080'), {'width': 1920, 'height': 1080})
self.assertEqual(parse_resolution('720p'), {'height': 720})
self.assertEqual(parse_resolution('4k'), {'height': 2160})
self.assertEqual(parse_resolution('8K'), {'height': 4320})
def test_parse_bitrate(self):
self.assertEqual(parse_bitrate(None), None)
self.assertEqual(parse_bitrate(''), None)
self.assertEqual(parse_bitrate('300kbps'), 300)
self.assertEqual(parse_bitrate('1500kbps'), 1500)
self.assertEqual(parse_bitrate('300 kbps'), 300)
def test_version_tuple(self):
self.assertEqual(version_tuple('1'), (1,))
self.assertEqual(version_tuple('10.23.344'), (10, 23, 344))
@ -994,6 +1201,18 @@ ffmpeg version 2.4.4 Copyright (c) 2000-2014 the FFmpeg ...'''), '2.4.4')
self.assertFalse(match_str(
'like_count > 100 & dislike_count <? 50 & description',
{'like_count': 190, 'dislike_count': 10}))
self.assertTrue(match_str('is_live', {'is_live': True}))
self.assertFalse(match_str('is_live', {'is_live': False}))
self.assertFalse(match_str('is_live', {'is_live': None}))
self.assertFalse(match_str('is_live', {}))
self.assertFalse(match_str('!is_live', {'is_live': True}))
self.assertTrue(match_str('!is_live', {'is_live': False}))
self.assertTrue(match_str('!is_live', {'is_live': None}))
self.assertTrue(match_str('!is_live', {}))
self.assertTrue(match_str('title', {'title': 'abc'}))
self.assertTrue(match_str('title', {'title': ''}))
self.assertFalse(match_str('!title', {'title': 'abc'}))
self.assertFalse(match_str('!title', {'title': ''}))
def test_parse_dfxp_time_expr(self):
self.assertEqual(parse_dfxp_time_expr(None), None)
@ -1017,7 +1236,7 @@ ffmpeg version 2.4.4 Copyright (c) 2000-2014 the FFmpeg ...'''), '2.4.4')
<p begin="3" dur="-1">Ignored, three</p>
</div>
</body>
</tt>'''
</tt>'''.encode('utf-8')
srt_data = '''1
00:00:00,000 --> 00:00:01,000
The following line contains Chinese characters and special symbols
@ -1042,7 +1261,7 @@ Line
<p begin="0" end="1">The first line</p>
</div>
</body>
</tt>'''
</tt>'''.encode('utf-8')
srt_data = '''1
00:00:00,000 --> 00:00:01,000
The first line
@ -1050,6 +1269,67 @@ The first line
'''
self.assertEqual(dfxp2srt(dfxp_data_no_default_namespace), srt_data)
dfxp_data_with_style = '''<?xml version="1.0" encoding="utf-8"?>
<tt xmlns="http://www.w3.org/2006/10/ttaf1" xmlns:ttp="http://www.w3.org/2006/10/ttaf1#parameter" ttp:timeBase="media" xmlns:tts="http://www.w3.org/2006/10/ttaf1#style" xml:lang="en" xmlns:ttm="http://www.w3.org/2006/10/ttaf1#metadata">
<head>
<styling>
<style id="s2" style="s0" tts:color="cyan" tts:fontWeight="bold" />
<style id="s1" style="s0" tts:color="yellow" tts:fontStyle="italic" />
<style id="s3" style="s0" tts:color="lime" tts:textDecoration="underline" />
<style id="s0" tts:backgroundColor="black" tts:fontStyle="normal" tts:fontSize="16" tts:fontFamily="sansSerif" tts:color="white" />
</styling>
</head>
<body tts:textAlign="center" style="s0">
<div>
<p begin="00:00:02.08" id="p0" end="00:00:05.84">default style<span tts:color="red">custom style</span></p>
<p style="s2" begin="00:00:02.08" id="p0" end="00:00:05.84"><span tts:color="lime">part 1<br /></span><span tts:color="cyan">part 2</span></p>
<p style="s3" begin="00:00:05.84" id="p1" end="00:00:09.56">line 3<br />part 3</p>
<p style="s1" tts:textDecoration="underline" begin="00:00:09.56" id="p2" end="00:00:12.36"><span style="s2" tts:color="lime">inner<br /> </span>style</p>
</div>
</body>
</tt>'''.encode('utf-8')
srt_data = '''1
00:00:02,080 --> 00:00:05,839
<font color="white" face="sansSerif" size="16">default style<font color="red">custom style</font></font>
2
00:00:02,080 --> 00:00:05,839
<b><font color="cyan" face="sansSerif" size="16"><font color="lime">part 1
</font>part 2</font></b>
3
00:00:05,839 --> 00:00:09,560
<u><font color="lime">line 3
part 3</font></u>
4
00:00:09,560 --> 00:00:12,359
<i><u><font color="yellow"><font color="lime">inner
</font>style</font></u></i>
'''
self.assertEqual(dfxp2srt(dfxp_data_with_style), srt_data)
dfxp_data_non_utf8 = '''<?xml version="1.0" encoding="UTF-16"?>
<tt xmlns="http://www.w3.org/ns/ttml" xml:lang="en" xmlns:tts="http://www.w3.org/ns/ttml#parameter">
<body>
<div xml:lang="en">
<p begin="0" end="1">Line 1</p>
<p begin="1" end="2">第二行</p>
</div>
</body>
</tt>'''.encode('utf-16')
srt_data = '''1
00:00:00,000 --> 00:00:01,000
Line 1
2
00:00:01,000 --> 00:00:02,000
第二行
'''
self.assertEqual(dfxp2srt(dfxp_data_non_utf8), srt_data)
def test_cli_option(self):
self.assertEqual(cli_option({'proxy': '127.0.0.1:3128'}, '--proxy', 'proxy'), ['--proxy', '127.0.0.1:3128'])
self.assertEqual(cli_option({'proxy': None}, '--proxy', 'proxy'), [])
@ -1095,6 +1375,10 @@ The first line
cli_bool_option(
{'nocheckcertificate': False}, '--check-certificate', 'nocheckcertificate', 'false', 'true', '='),
['--check-certificate=true'])
self.assertEqual(
cli_bool_option(
{}, '--check-certificate', 'nocheckcertificate', 'false', 'true', '='),
[])
def test_ohdave_rsa_encrypt(self):
N = 0xab86b6371b5318aaa1d3c9e612a9f1264f372323c8c0f19875b5fc3b3fd3afcc1e5bec527aa94bfa85bffc157e4245aebda05389a5357b75115ac94f074aefcd
@ -1104,6 +1388,14 @@ The first line
ohdave_rsa_encrypt(b'aa111222', e, N),
'726664bd9a23fd0c70f9f1b84aab5e3905ce1e45a584e9cbcf9bcc7510338fc1986d6c599ff990d923aa43c51c0d9013cd572e13bc58f4ae48f2ed8c0b0ba881')
def test_pkcs1pad(self):
data = [1, 2, 3]
padded_data = pkcs1pad(data, 32)
self.assertEqual(padded_data[:2], [0, 2])
self.assertEqual(padded_data[28:], [0, 1, 2, 3])
self.assertRaises(ValueError, pkcs1pad, data, 8)
def test_encode_base_n(self):
self.assertEqual(encode_base_n(0, 30), '0')
self.assertEqual(encode_base_n(80, 30), '2k')
@ -1115,6 +1407,20 @@ The first line
self.assertRaises(ValueError, encode_base_n, 0, 70)
self.assertRaises(ValueError, encode_base_n, 0, 60, custom_table)
def test_caesar(self):
self.assertEqual(caesar('ace', 'abcdef', 2), 'cea')
self.assertEqual(caesar('cea', 'abcdef', -2), 'ace')
self.assertEqual(caesar('ace', 'abcdef', -2), 'eac')
self.assertEqual(caesar('eac', 'abcdef', 2), 'ace')
self.assertEqual(caesar('ace', 'abcdef', 0), 'ace')
self.assertEqual(caesar('xyz', 'abcdef', 2), 'xyz')
self.assertEqual(caesar('abc', 'acegik', 2), 'ebg')
self.assertEqual(caesar('ebg', 'acegik', -2), 'abc')
def test_rot47(self):
self.assertEqual(rot47('youtube-dl'), r'J@FEF36\5=')
self.assertEqual(rot47('YOUTUBE-DL'), r'*~&%&qt\s{')
def test_urshift(self):
self.assertEqual(urshift(3, 1), 1)
self.assertEqual(urshift(-3, 1), 2147483646)
@ -1136,6 +1442,12 @@ The first line
self.assertEqual(get_element_by_attribute('class', 'foo', html), None)
self.assertEqual(get_element_by_attribute('class', 'no-such-foo', html), None)
html = '''
<div itemprop="author" itemscope>foo</div>
'''
self.assertEqual(get_element_by_attribute('itemprop', 'author', html), 'foo')
def test_get_elements_by_class(self):
html = '''
<span class="foo bar">nice</span><span class="foo bar">also nice</span>

View File

@ -0,0 +1,275 @@
#!/usr/bin/env python
# coding: utf-8
from __future__ import unicode_literals
# Allow direct execution
import os
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from test.helper import expect_value
from youtube_dl.extractor import YoutubeIE
class TestYoutubeChapters(unittest.TestCase):
_TEST_CASES = [
(
# https://www.youtube.com/watch?v=A22oy8dFjqc
# pattern: 00:00 - <title>
'''This is the absolute ULTIMATE experience of Queen's set at LIVE AID, this is the best video mixed to the absolutely superior stereo radio broadcast. This vastly superior audio mix takes a huge dump on all of the official mixes. Best viewed in 1080p. ENJOY! ***MAKE SURE TO READ THE DESCRIPTION***<br /><a href="#" onclick="yt.www.watch.player.seekTo(00*60+36);return false;">00:36</a> - Bohemian Rhapsody<br /><a href="#" onclick="yt.www.watch.player.seekTo(02*60+42);return false;">02:42</a> - Radio Ga Ga<br /><a href="#" onclick="yt.www.watch.player.seekTo(06*60+53);return false;">06:53</a> - Ay Oh!<br /><a href="#" onclick="yt.www.watch.player.seekTo(07*60+34);return false;">07:34</a> - Hammer To Fall<br /><a href="#" onclick="yt.www.watch.player.seekTo(12*60+08);return false;">12:08</a> - Crazy Little Thing Called Love<br /><a href="#" onclick="yt.www.watch.player.seekTo(16*60+03);return false;">16:03</a> - We Will Rock You<br /><a href="#" onclick="yt.www.watch.player.seekTo(17*60+18);return false;">17:18</a> - We Are The Champions<br /><a href="#" onclick="yt.www.watch.player.seekTo(21*60+12);return false;">21:12</a> - Is This The World We Created...?<br /><br />Short song analysis:<br /><br />- "Bohemian Rhapsody": Although it's a short medley version, it's one of the best performances of the ballad section, with Freddie nailing the Bb4s with the correct studio phrasing (for the first time ever!).<br /><br />- "Radio Ga Ga": Although it's missing one chorus, this is one of - if not the best - the best versions ever, Freddie nails all the Bb4s and sounds very clean! Spike Edney's Roland Jupiter 8 also really shines through on this mix, compared to the DVD releases!<br /><br />- "Audience Improv": A great improv, Freddie sounds strong and confident. You gotta love when he sustains that A4 for 4 seconds!<br /><br />- "Hammer To Fall": Despite missing a verse and a chorus, it's a strong version (possibly the best ever). Freddie sings the song amazingly, and even ad-libs a C#5 and a C5! Also notice how heavy Brian's guitar sounds compared to the thin DVD mixes - it roars!<br /><br />- "Crazy Little Thing Called Love": A great version, the crowd loves the song, the jam is great as well! Only downside to this is the slight feedback issues.<br /><br />- "We Will Rock You": Although cut down to the 1st verse and chorus, Freddie sounds strong. He nails the A4, and the solo from Dr. May is brilliant!<br /><br />- "We Are the Champions": Perhaps the high-light of the performance - Freddie is very daring on this version, he sustains the pre-chorus Bb4s, nails the 1st C5, belts great A4s, but most importantly: He nails the chorus Bb4s, in all 3 choruses! This is the only time he has ever done so! It has to be said though, the last one sounds a bit rough, but that's a side effect of belting high notes for the past 18 minutes, with nodules AND laryngitis!<br /><br />- "Is This The World We Created... ?": Freddie and Brian perform a beautiful version of this, and it is one of the best versions ever. It's both sad and hilarious that a couple of BBC engineers are talking over the song, one of them being completely oblivious of the fact that he is interrupting the performance, on live television... Which was being televised to almost 2 billion homes.<br /><br /><br />All rights go to their respective owners!<br />-----Copyright Disclaimer Under Section 107 of the Copyright Act 1976, allowance is made for fair use for purposes such as criticism, comment, news reporting, teaching, scholarship, and research. Fair use is a use permitted by copyright statute that might otherwise be infringing. Non-profit, educational or personal use tips the balance in favor of fair use''',
1477,
[{
'start_time': 36,
'end_time': 162,
'title': 'Bohemian Rhapsody',
}, {
'start_time': 162,
'end_time': 413,
'title': 'Radio Ga Ga',
}, {
'start_time': 413,
'end_time': 454,
'title': 'Ay Oh!',
}, {
'start_time': 454,
'end_time': 728,
'title': 'Hammer To Fall',
}, {
'start_time': 728,
'end_time': 963,
'title': 'Crazy Little Thing Called Love',
}, {
'start_time': 963,
'end_time': 1038,
'title': 'We Will Rock You',
}, {
'start_time': 1038,
'end_time': 1272,
'title': 'We Are The Champions',
}, {
'start_time': 1272,
'end_time': 1477,
'title': 'Is This The World We Created...?',
}]
),
(
# https://www.youtube.com/watch?v=ekYlRhALiRQ
# pattern: <num>. <title> 0:00
'1. Those Beaten Paths of Confusion <a href="#" onclick="yt.www.watch.player.seekTo(0*60+00);return false;">0:00</a><br />2. Beyond the Shadows of Emptiness & Nothingness <a href="#" onclick="yt.www.watch.player.seekTo(11*60+47);return false;">11:47</a><br />3. Poison Yourself...With Thought <a href="#" onclick="yt.www.watch.player.seekTo(26*60+30);return false;">26:30</a><br />4. The Agents of Transformation <a href="#" onclick="yt.www.watch.player.seekTo(35*60+57);return false;">35:57</a><br />5. Drowning in the Pain of Consciousness <a href="#" onclick="yt.www.watch.player.seekTo(44*60+32);return false;">44:32</a><br />6. Deny the Disease of Life <a href="#" onclick="yt.www.watch.player.seekTo(53*60+07);return false;">53:07</a><br /><br />More info/Buy: http://crepusculonegro.storenvy.com/products/257645-cn-03-arizmenda-within-the-vacuum-of-infinity<br /><br />No copyright is intended. The rights to this video are assumed by the owner and its affiliates.',
4009,
[{
'start_time': 0,
'end_time': 707,
'title': '1. Those Beaten Paths of Confusion',
}, {
'start_time': 707,
'end_time': 1590,
'title': '2. Beyond the Shadows of Emptiness & Nothingness',
}, {
'start_time': 1590,
'end_time': 2157,
'title': '3. Poison Yourself...With Thought',
}, {
'start_time': 2157,
'end_time': 2672,
'title': '4. The Agents of Transformation',
}, {
'start_time': 2672,
'end_time': 3187,
'title': '5. Drowning in the Pain of Consciousness',
}, {
'start_time': 3187,
'end_time': 4009,
'title': '6. Deny the Disease of Life',
}]
),
(
# https://www.youtube.com/watch?v=WjL4pSzog9w
# pattern: 00:00 <title>
'<a href="https://arizmenda.bandcamp.com/merch/despairs-depths-descended-cd" class="yt-uix-servicelink " data-target-new-window="True" data-servicelink="CDAQ6TgYACITCNf1raqT2dMCFdRjGAod_o0CBSj4HQ" data-url="https://arizmenda.bandcamp.com/merch/despairs-depths-descended-cd" rel="nofollow noopener" target="_blank">https://arizmenda.bandcamp.com/merch/...</a><br /><br /><a href="#" onclick="yt.www.watch.player.seekTo(00*60+00);return false;">00:00</a> Christening Unborn Deformities <br /><a href="#" onclick="yt.www.watch.player.seekTo(07*60+08);return false;">07:08</a> Taste of Purity<br /><a href="#" onclick="yt.www.watch.player.seekTo(16*60+16);return false;">16:16</a> Sculpting Sins of a Universal Tongue<br /><a href="#" onclick="yt.www.watch.player.seekTo(24*60+45);return false;">24:45</a> Birth<br /><a href="#" onclick="yt.www.watch.player.seekTo(31*60+24);return false;">31:24</a> Neves<br /><a href="#" onclick="yt.www.watch.player.seekTo(37*60+55);return false;">37:55</a> Libations in Limbo',
2705,
[{
'start_time': 0,
'end_time': 428,
'title': 'Christening Unborn Deformities',
}, {
'start_time': 428,
'end_time': 976,
'title': 'Taste of Purity',
}, {
'start_time': 976,
'end_time': 1485,
'title': 'Sculpting Sins of a Universal Tongue',
}, {
'start_time': 1485,
'end_time': 1884,
'title': 'Birth',
}, {
'start_time': 1884,
'end_time': 2275,
'title': 'Neves',
}, {
'start_time': 2275,
'end_time': 2705,
'title': 'Libations in Limbo',
}]
),
(
# https://www.youtube.com/watch?v=o3r1sn-t3is
# pattern: <title> 00:00 <note>
'Download this show in MP3: <a href="http://sh.st/njZKK" class="yt-uix-servicelink " data-url="http://sh.st/njZKK" data-target-new-window="True" data-servicelink="CDAQ6TgYACITCK3j8_6o2dMCFVDCGAoduVAKKij4HQ" rel="nofollow noopener" target="_blank">http://sh.st/njZKK</a><br /><br />Setlist:<br />I-E-A-I-A-I-O <a href="#" onclick="yt.www.watch.player.seekTo(00*60+45);return false;">00:45</a><br />Suite-Pee <a href="#" onclick="yt.www.watch.player.seekTo(4*60+26);return false;">4:26</a> (Incomplete)<br />Attack <a href="#" onclick="yt.www.watch.player.seekTo(5*60+31);return false;">5:31</a> (First live performance since 2011)<br />Prison Song <a href="#" onclick="yt.www.watch.player.seekTo(8*60+42);return false;">8:42</a><br />Know <a href="#" onclick="yt.www.watch.player.seekTo(12*60+32);return false;">12:32</a> (First live performance since 2011)<br />Aerials <a href="#" onclick="yt.www.watch.player.seekTo(15*60+32);return false;">15:32</a><br />Soldier Side - Intro <a href="#" onclick="yt.www.watch.player.seekTo(19*60+13);return false;">19:13</a><br />B.Y.O.B. <a href="#" onclick="yt.www.watch.player.seekTo(20*60+09);return false;">20:09</a><br />Soil <a href="#" onclick="yt.www.watch.player.seekTo(24*60+32);return false;">24:32</a><br />Darts <a href="#" onclick="yt.www.watch.player.seekTo(27*60+48);return false;">27:48</a><br />Radio/Video <a href="#" onclick="yt.www.watch.player.seekTo(30*60+38);return false;">30:38</a><br />Hypnotize <a href="#" onclick="yt.www.watch.player.seekTo(35*60+05);return false;">35:05</a><br />Temper <a href="#" onclick="yt.www.watch.player.seekTo(38*60+08);return false;">38:08</a> (First live performance since 1999)<br />CUBErt <a href="#" onclick="yt.www.watch.player.seekTo(41*60+00);return false;">41:00</a><br />Needles <a href="#" onclick="yt.www.watch.player.seekTo(42*60+57);return false;">42:57</a><br />Deer Dance <a href="#" onclick="yt.www.watch.player.seekTo(46*60+27);return false;">46:27</a><br />Bounce <a href="#" onclick="yt.www.watch.player.seekTo(49*60+38);return false;">49:38</a><br />Suggestions <a href="#" onclick="yt.www.watch.player.seekTo(51*60+25);return false;">51:25</a><br />Psycho <a href="#" onclick="yt.www.watch.player.seekTo(53*60+52);return false;">53:52</a><br />Chop Suey! <a href="#" onclick="yt.www.watch.player.seekTo(58*60+13);return false;">58:13</a><br />Lonely Day <a href="#" onclick="yt.www.watch.player.seekTo(1*3600+01*60+15);return false;">1:01:15</a><br />Question! <a href="#" onclick="yt.www.watch.player.seekTo(1*3600+04*60+14);return false;">1:04:14</a><br />Lost in Hollywood <a href="#" onclick="yt.www.watch.player.seekTo(1*3600+08*60+10);return false;">1:08:10</a><br />Vicinity of Obscenity <a href="#" onclick="yt.www.watch.player.seekTo(1*3600+13*60+40);return false;">1:13:40</a>(First live performance since 2012)<br />Forest <a href="#" onclick="yt.www.watch.player.seekTo(1*3600+16*60+17);return false;">1:16:17</a><br />Cigaro <a href="#" onclick="yt.www.watch.player.seekTo(1*3600+20*60+02);return false;">1:20:02</a><br />Toxicity <a href="#" onclick="yt.www.watch.player.seekTo(1*3600+23*60+57);return false;">1:23:57</a>(with Chino Moreno)<br />Sugar <a href="#" onclick="yt.www.watch.player.seekTo(1*3600+27*60+53);return false;">1:27:53</a>',
5640,
[{
'start_time': 45,
'end_time': 266,
'title': 'I-E-A-I-A-I-O',
}, {
'start_time': 266,
'end_time': 331,
'title': 'Suite-Pee (Incomplete)',
}, {
'start_time': 331,
'end_time': 522,
'title': 'Attack (First live performance since 2011)',
}, {
'start_time': 522,
'end_time': 752,
'title': 'Prison Song',
}, {
'start_time': 752,
'end_time': 932,
'title': 'Know (First live performance since 2011)',
}, {
'start_time': 932,
'end_time': 1153,
'title': 'Aerials',
}, {
'start_time': 1153,
'end_time': 1209,
'title': 'Soldier Side - Intro',
}, {
'start_time': 1209,
'end_time': 1472,
'title': 'B.Y.O.B.',
}, {
'start_time': 1472,
'end_time': 1668,
'title': 'Soil',
}, {
'start_time': 1668,
'end_time': 1838,
'title': 'Darts',
}, {
'start_time': 1838,
'end_time': 2105,
'title': 'Radio/Video',
}, {
'start_time': 2105,
'end_time': 2288,
'title': 'Hypnotize',
}, {
'start_time': 2288,
'end_time': 2460,
'title': 'Temper (First live performance since 1999)',
}, {
'start_time': 2460,
'end_time': 2577,
'title': 'CUBErt',
}, {
'start_time': 2577,
'end_time': 2787,
'title': 'Needles',
}, {
'start_time': 2787,
'end_time': 2978,
'title': 'Deer Dance',
}, {
'start_time': 2978,
'end_time': 3085,
'title': 'Bounce',
}, {
'start_time': 3085,
'end_time': 3232,
'title': 'Suggestions',
}, {
'start_time': 3232,
'end_time': 3493,
'title': 'Psycho',
}, {
'start_time': 3493,
'end_time': 3675,
'title': 'Chop Suey!',
}, {
'start_time': 3675,
'end_time': 3854,
'title': 'Lonely Day',
}, {
'start_time': 3854,
'end_time': 4090,
'title': 'Question!',
}, {
'start_time': 4090,
'end_time': 4420,
'title': 'Lost in Hollywood',
}, {
'start_time': 4420,
'end_time': 4577,
'title': 'Vicinity of Obscenity (First live performance since 2012)',
}, {
'start_time': 4577,
'end_time': 4802,
'title': 'Forest',
}, {
'start_time': 4802,
'end_time': 5037,
'title': 'Cigaro',
}, {
'start_time': 5037,
'end_time': 5273,
'title': 'Toxicity (with Chino Moreno)',
}, {
'start_time': 5273,
'end_time': 5640,
'title': 'Sugar',
}]
),
(
# https://www.youtube.com/watch?v=PkYLQbsqCE8
# pattern: <num> - <title> [<latinized title>] 0:00:00
'''Затемно (Zatemno) is an Obscure Black Metal Band from Russia.<br /><br />"Во прах (Vo prakh)'' Into The Ashes", Debut mini-album released may 6, 2016, by Death Knell Productions<br />Released on 6 panel digipak CD, limited to 100 copies only<br />And digital format on Bandcamp<br /><br />Tracklist<br /><br />1 - Во прах [Vo prakh] <a href="#" onclick="yt.www.watch.player.seekTo(0*3600+00*60+00);return false;">0:00:00</a><br />2 - Искупление [Iskupleniye] <a href="#" onclick="yt.www.watch.player.seekTo(0*3600+08*60+10);return false;">0:08:10</a><br />3 - Из серпов луны...[Iz serpov luny] <a href="#" onclick="yt.www.watch.player.seekTo(0*3600+14*60+30);return false;">0:14:30</a><br /><br />Links:<br /><a href="https://deathknellprod.bandcamp.com/album/--2" class="yt-uix-servicelink " data-target-new-window="True" data-url="https://deathknellprod.bandcamp.com/album/--2" data-servicelink="CC8Q6TgYACITCNP234Kr2dMCFcNxGAodQqsIwSj4HQ" target="_blank" rel="nofollow noopener">https://deathknellprod.bandcamp.com/a...</a><br /><a href="https://www.facebook.com/DeathKnellProd/" class="yt-uix-servicelink " data-target-new-window="True" data-url="https://www.facebook.com/DeathKnellProd/" data-servicelink="CC8Q6TgYACITCNP234Kr2dMCFcNxGAodQqsIwSj4HQ" target="_blank" rel="nofollow noopener">https://www.facebook.com/DeathKnellProd/</a><br /><br /><br />I don't have any right about this artifact, my only intention is to spread the music of the band, all rights are reserved to the Затемно (Zatemno) and his producers, Death Knell Productions.<br /><br />------------------------------------------------------------------<br /><br />Subscribe for more videos like this.<br />My link: <a href="https://web.facebook.com/AttackOfTheDragons" class="yt-uix-servicelink " data-target-new-window="True" data-url="https://web.facebook.com/AttackOfTheDragons" data-servicelink="CC8Q6TgYACITCNP234Kr2dMCFcNxGAodQqsIwSj4HQ" target="_blank" rel="nofollow noopener">https://web.facebook.com/AttackOfTheD...</a>''',
1138,
[{
'start_time': 0,
'end_time': 490,
'title': '1 - Во прах [Vo prakh]',
}, {
'start_time': 490,
'end_time': 870,
'title': '2 - Искупление [Iskupleniye]',
}, {
'start_time': 870,
'end_time': 1138,
'title': '3 - Из серпов луны...[Iz serpov luny]',
}]
),
(
# https://www.youtube.com/watch?v=xZW70zEasOk
# time point more than duration
'''● LCS Spring finals: Saturday and Sunday from <a href="#" onclick="yt.www.watch.player.seekTo(13*60+30);return false;">13:30</a> outside the venue! <br />● PAX East: Fri, Sat & Sun - more info in tomorrows video on the main channel!''',
283,
[]
),
]
def test_youtube_chapters(self):
for description, duration, expected_chapters in self._TEST_CASES:
ie = YoutubeIE()
expect_value(
self, ie._extract_chapters_from_description(description, duration),
expected_chapters, None)
if __name__ == '__main__':
unittest.main()

View File

@ -61,7 +61,7 @@ class TestYoutubeLists(unittest.TestCase):
dl = FakeYDL()
dl.params['extract_flat'] = True
ie = YoutubePlaylistIE(dl)
result = ie.extract('https://www.youtube.com/playlist?list=PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re')
result = ie.extract('https://www.youtube.com/playlist?list=PL-KKIb8rvtMSrAO9YFbeM6UQrAqoFTUWv')
self.assertIsPlaylist(result)
for entry in result['entries']:
self.assertTrue(entry.get('title'))

View File

@ -74,6 +74,28 @@ _TESTS = [
]
class TestPlayerInfo(unittest.TestCase):
def test_youtube_extract_player_info(self):
PLAYER_URLS = (
('https://www.youtube.com/s/player/64dddad9/player_ias.vflset/en_US/base.js', '64dddad9'),
# obsolete
('https://www.youtube.com/yts/jsbin/player_ias-vfle4-e03/en_US/base.js', 'vfle4-e03'),
('https://www.youtube.com/yts/jsbin/player_ias-vfl49f_g4/en_US/base.js', 'vfl49f_g4'),
('https://www.youtube.com/yts/jsbin/player_ias-vflCPQUIL/en_US/base.js', 'vflCPQUIL'),
('https://www.youtube.com/yts/jsbin/player-vflzQZbt7/en_US/base.js', 'vflzQZbt7'),
('https://www.youtube.com/yts/jsbin/player-en_US-vflaxXRn1/base.js', 'vflaxXRn1'),
('https://s.ytimg.com/yts/jsbin/html5player-en_US-vflXGBaUN.js', 'vflXGBaUN'),
('https://s.ytimg.com/yts/jsbin/html5player-en_US-vflKjOTVq/html5player.js', 'vflKjOTVq'),
('http://s.ytimg.com/yt/swfbin/watch_as3-vflrEm9Nq.swf', 'vflrEm9Nq'),
('https://s.ytimg.com/yts/swfbin/player-vflenCdZL/watch_as3.swf', 'vflenCdZL'),
)
for player_url, expected_player_id in PLAYER_URLS:
expected_player_type = player_url.split('.')[-1]
player_type, player_id = YoutubeIE._extract_player_info(player_url)
self.assertEqual(player_type, expected_player_type)
self.assertEqual(player_id, expected_player_id)
class TestSignature(unittest.TestCase):
def setUp(self):
TEST_DIR = os.path.dirname(os.path.abspath(__file__))

View File

@ -0,0 +1,6 @@
# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This is a generated file! Do not edit.
#HttpOnly_www.foobar.foobar FALSE / TRUE 2147483647 HTTPONLY_COOKIE HTTPONLY_COOKIE_VALUE
www.foobar.foobar FALSE / TRUE 2147483647 JS_ACCESSIBLE_COOKIE JS_ACCESSIBLE_COOKIE_VALUE

View File

@ -0,0 +1,9 @@
# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This is a generated file! Do not edit.
# Cookie file entry with invalid number of fields - 6 instead of 7
www.foobar.foobar FALSE / FALSE 0 COOKIE
# Cookie file entry with invalid expires at
www.foobar.foobar FALSE / FALSE 1.7976931348623157e+308 COOKIE VALUE

View File

@ -0,0 +1,6 @@
# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This is a generated file! Do not edit.
www.foobar.foobar FALSE / TRUE YoutubeDLExpiresEmpty YoutubeDLExpiresEmptyValue
www.foobar.foobar FALSE / TRUE 0 YoutubeDLExpires0 YoutubeDLExpires0Value

10
test/testdata/f4m/custom_base_url.f4m vendored Normal file
View File

@ -0,0 +1,10 @@
<?xml version="1.0" encoding="UTF-8"?>
<manifest xmlns="http://ns.adobe.com/f4m/1.0">
<streamType>recorded</streamType>
<baseURL>http://vod.livestream.com/events/0000000000673980/</baseURL>
<duration>269.293</duration>
<bootstrapInfo profile="named" id="bootstrap_1">AAAAm2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAEG+0AAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAAC4BAAAAVmFmcnQAAAAAAAAD6AAAAAAEAAAAAQAAAAAAAAAAAAAXcAAAAC0AAAAAAAQHQAAAE5UAAAAuAAAAAAAEGtUAAAEYAAAAAAAAAAAAAAAAAAAAAAA=</bootstrapInfo>
<media url="b90f532f-b0f6-4f4e-8289-706d490b2fd8_2292" bootstrapInfoId="bootstrap_1" bitrate="2148" width="1280" height="720" videoCodec="avc1.4d401f" audioCodec="mp4a.40.2">
<metadata>AgAKb25NZXRhRGF0YQgAAAAIAAhkdXJhdGlvbgBAcNSwIMSbpgAFd2lkdGgAQJQAAAAAAAAABmhlaWdodABAhoAAAAAAAAAJZnJhbWVyYXRlAEA4/7DoLwW3AA12aWRlb2RhdGFyYXRlAECe1DLgjcobAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGSimlvaPKQADGF1ZGlvY29kZWNpZABAJAAAAAAAAAAACQ==</metadata>
</media>
</manifest>

View File

@ -0,0 +1,14 @@
#EXTM3U
#EXT-X-VERSION:5
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Francais",DEFAULT=NO,FORCED=NO,URI="http://replayftv-pmd.francetv.fr/subtitles/2017/16/156589847-1492488987.m3u8",LANGUAGE="fra"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac",LANGUAGE="fra",NAME="Francais",DEFAULT=YES, AUTOSELECT=YES
#EXT-X-STREAM-INF:SUBTITLES="subs",AUDIO="aac",PROGRAM-ID=1,BANDWIDTH=180000,RESOLUTION=256x144,CODECS="avc1.66.30, mp4a.40.2"
http://replayftv-vh.akamaihd.net/i/streaming-adaptatif_france-dom-tom/2017/S16/J2/156589847-58f59130c1f52-,standard1,standard2,standard3,standard4,standard5,.mp4.csmil/index_0_av.m3u8?null=0
#EXT-X-STREAM-INF:SUBTITLES="subs",AUDIO="aac",PROGRAM-ID=1,BANDWIDTH=303000,RESOLUTION=320x180,CODECS="avc1.66.30, mp4a.40.2"
http://replayftv-vh.akamaihd.net/i/streaming-adaptatif_france-dom-tom/2017/S16/J2/156589847-58f59130c1f52-,standard1,standard2,standard3,standard4,standard5,.mp4.csmil/index_1_av.m3u8?null=0
#EXT-X-STREAM-INF:SUBTITLES="subs",AUDIO="aac",PROGRAM-ID=1,BANDWIDTH=575000,RESOLUTION=512x288,CODECS="avc1.66.30, mp4a.40.2"
http://replayftv-vh.akamaihd.net/i/streaming-adaptatif_france-dom-tom/2017/S16/J2/156589847-58f59130c1f52-,standard1,standard2,standard3,standard4,standard5,.mp4.csmil/index_2_av.m3u8?null=0
#EXT-X-STREAM-INF:SUBTITLES="subs",AUDIO="aac",PROGRAM-ID=1,BANDWIDTH=831000,RESOLUTION=704x396,CODECS="avc1.77.30, mp4a.40.2"

16
test/testdata/m3u8/teamcoco_11995.m3u8 vendored Normal file
View File

@ -0,0 +1,16 @@
#EXTM3U
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-0",NAME="Default",AUTOSELECT=YES,DEFAULT=YES,URI="hls/CONAN_020217_Highlight_show-audio-160k_v4.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-1",NAME="Default",AUTOSELECT=YES,DEFAULT=YES,URI="hls/CONAN_020217_Highlight_show-audio-64k_v4.m3u8"
#EXT-X-I-FRAME-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=37862000,CODECS="avc1.4d001f",URI="hls/CONAN_020217_Highlight_show-2m_iframe.m3u8"
#EXT-X-I-FRAME-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=18750000,CODECS="avc1.4d001e",URI="hls/CONAN_020217_Highlight_show-1m_iframe.m3u8"
#EXT-X-I-FRAME-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=6535000,CODECS="avc1.42001e",URI="hls/CONAN_020217_Highlight_show-400k_iframe.m3u8"
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2374000,RESOLUTION=1024x576,CODECS="avc1.4d001f,mp4a.40.2",AUDIO="audio-0"
hls/CONAN_020217_Highlight_show-2m_v4.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1205000,RESOLUTION=640x360,CODECS="avc1.4d001e,mp4a.40.2",AUDIO="audio-0"
hls/CONAN_020217_Highlight_show-1m_v4.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=522000,RESOLUTION=400x224,CODECS="avc1.42001e,mp4a.40.2",AUDIO="audio-0"
hls/CONAN_020217_Highlight_show-400k_v4.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=413000,RESOLUTION=400x224,CODECS="avc1.42001e,mp4a.40.5",AUDIO="audio-1"
hls/CONAN_020217_Highlight_show-400k_v4.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=71000,CODECS="mp4a.40.5",AUDIO="audio-1"
hls/CONAN_020217_Highlight_show-audio-64k_v4.m3u8

28
test/testdata/m3u8/ted_18923.m3u8 vendored Normal file
View File

@ -0,0 +1,28 @@
#EXTM3U
#EXT-X-VERSION:4
#EXT-X-STREAM-INF:AUDIO="600k",BANDWIDTH=1255659,PROGRAM-ID=1,CODECS="avc1.42c01e,mp4a.40.2",RESOLUTION=640x360
/videos/BorisHesser_2018S/video/600k.m3u8?nobumpers=true&uniqueId=76011e2b
#EXT-X-STREAM-INF:AUDIO="600k",BANDWIDTH=163154,PROGRAM-ID=1,CODECS="avc1.42c00c,mp4a.40.2",RESOLUTION=320x180
/videos/BorisHesser_2018S/video/64k.m3u8?nobumpers=true&uniqueId=76011e2b
#EXT-X-STREAM-INF:AUDIO="600k",BANDWIDTH=481701,PROGRAM-ID=1,CODECS="avc1.42c015,mp4a.40.2",RESOLUTION=512x288
/videos/BorisHesser_2018S/video/180k.m3u8?nobumpers=true&uniqueId=76011e2b
#EXT-X-STREAM-INF:AUDIO="600k",BANDWIDTH=769968,PROGRAM-ID=1,CODECS="avc1.42c015,mp4a.40.2",RESOLUTION=512x288
/videos/BorisHesser_2018S/video/320k.m3u8?nobumpers=true&uniqueId=76011e2b
#EXT-X-STREAM-INF:AUDIO="600k",BANDWIDTH=984037,PROGRAM-ID=1,CODECS="avc1.42c015,mp4a.40.2",RESOLUTION=512x288
/videos/BorisHesser_2018S/video/450k.m3u8?nobumpers=true&uniqueId=76011e2b
#EXT-X-STREAM-INF:AUDIO="600k",BANDWIDTH=1693925,PROGRAM-ID=1,CODECS="avc1.4d401f,mp4a.40.2",RESOLUTION=853x480
/videos/BorisHesser_2018S/video/950k.m3u8?nobumpers=true&uniqueId=76011e2b
#EXT-X-STREAM-INF:AUDIO="600k",BANDWIDTH=2462469,PROGRAM-ID=1,CODECS="avc1.640028,mp4a.40.2",RESOLUTION=1280x720
/videos/BorisHesser_2018S/video/1500k.m3u8?nobumpers=true&uniqueId=76011e2b
#EXT-X-STREAM-INF:AUDIO="600k",BANDWIDTH=68101,PROGRAM-ID=1,CODECS="mp4a.40.2",DEFAULT=YES
/videos/BorisHesser_2018S/audio/600k.m3u8?nobumpers=true&uniqueId=76011e2b
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=74298,PROGRAM-ID=1,CODECS="avc1.42c00c",RESOLUTION=320x180,URI="/videos/BorisHesser_2018S/video/64k_iframe.m3u8?nobumpers=true&uniqueId=76011e2b"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=216200,PROGRAM-ID=1,CODECS="avc1.42c015",RESOLUTION=512x288,URI="/videos/BorisHesser_2018S/video/180k_iframe.m3u8?nobumpers=true&uniqueId=76011e2b"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=304717,PROGRAM-ID=1,CODECS="avc1.42c015",RESOLUTION=512x288,URI="/videos/BorisHesser_2018S/video/320k_iframe.m3u8?nobumpers=true&uniqueId=76011e2b"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=350933,PROGRAM-ID=1,CODECS="avc1.42c015",RESOLUTION=512x288,URI="/videos/BorisHesser_2018S/video/450k_iframe.m3u8?nobumpers=true&uniqueId=76011e2b"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=495850,PROGRAM-ID=1,CODECS="avc1.42c01e",RESOLUTION=640x360,URI="/videos/BorisHesser_2018S/video/600k_iframe.m3u8?nobumpers=true&uniqueId=76011e2b"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=810750,PROGRAM-ID=1,CODECS="avc1.4d401f",RESOLUTION=853x480,URI="/videos/BorisHesser_2018S/video/950k_iframe.m3u8?nobumpers=true&uniqueId=76011e2b"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=1273700,PROGRAM-ID=1,CODECS="avc1.640028",RESOLUTION=1280x720,URI="/videos/BorisHesser_2018S/video/1500k_iframe.m3u8?nobumpers=true&uniqueId=76011e2b"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="600k",LANGUAGE="en",NAME="Audio",AUTOSELECT=YES,DEFAULT=YES,URI="/videos/BorisHesser_2018S/audio/600k.m3u8?nobumpers=true&uniqueId=76011e2b",BANDWIDTH=614400

View File

@ -0,0 +1,13 @@
#EXTM3U
#EXT-X-VERSION:4
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio",LANGUAGE="eng",NAME="English",URI="http://k.toggle.sg/fhls/p/2082311/sp/208231100/serveFlavor/entryId/0_89q6e8ku/v/2/pv/1/flavorId/0_sa2ntrdg/name/a.mp4/index.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio",LANGUAGE="und",NAME="Undefined",URI="http://k.toggle.sg/fhls/p/2082311/sp/208231100/serveFlavor/entryId/0_89q6e8ku/v/2/pv/1/flavorId/0_r7y0nitg/name/a.mp4/index.m3u8"
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=155648,RESOLUTION=320x180,AUDIO="audio"
http://k.toggle.sg/fhls/p/2082311/sp/208231100/serveFlavor/entryId/0_89q6e8ku/v/2/pv/1/flavorId/0_qlk9hlzr/name/a.mp4/index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=502784,RESOLUTION=480x270,AUDIO="audio"
http://k.toggle.sg/fhls/p/2082311/sp/208231100/serveFlavor/entryId/0_89q6e8ku/v/2/pv/1/flavorId/0_oefackmi/name/a.mp4/index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=827392,RESOLUTION=640x360,AUDIO="audio"
http://k.toggle.sg/fhls/p/2082311/sp/208231100/serveFlavor/entryId/0_89q6e8ku/v/12/pv/1/flavorId/0_vyg9pj7k/name/a.mp4/index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1396736,RESOLUTION=854x480,AUDIO="audio"
http://k.toggle.sg/fhls/p/2082311/sp/208231100/serveFlavor/entryId/0_89q6e8ku/v/12/pv/1/flavorId/0_50n4psvx/name/a.mp4/index.m3u8

20
test/testdata/m3u8/twitch_vod.m3u8 vendored Normal file
View File

@ -0,0 +1,20 @@
#EXTM3U
#EXT-X-TWITCH-INFO:ORIGIN="s3",CLUSTER="edgecast_vod",REGION="EU",MANIFEST-CLUSTER="edgecast_vod",USER-IP="109.171.17.81"
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="chunked",NAME="Source",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=3214134,CODECS="avc1.100.31,mp4a.40.2",RESOLUTION="1280x720",VIDEO="chunked"
https://vod.edgecast.hls.ttvnw.net/e5da31ab49_riotgames_15001215120_261543898/chunked/index-muted-HM49I092CC.m3u8
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="high",NAME="High",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1603789,CODECS="avc1.42C01F,mp4a.40.2",RESOLUTION="1280x720",VIDEO="high"
https://vod.edgecast.hls.ttvnw.net/e5da31ab49_riotgames_15001215120_261543898/high/index-muted-HM49I092CC.m3u8
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="medium",NAME="Medium",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=893387,CODECS="avc1.42C01E,mp4a.40.2",RESOLUTION="852x480",VIDEO="medium"
https://vod.edgecast.hls.ttvnw.net/e5da31ab49_riotgames_15001215120_261543898/medium/index-muted-HM49I092CC.m3u8
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="low",NAME="Low",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=628347,CODECS="avc1.42C01E,mp4a.40.2",RESOLUTION="640x360",VIDEO="low"
https://vod.edgecast.hls.ttvnw.net/e5da31ab49_riotgames_15001215120_261543898/low/index-muted-HM49I092CC.m3u8
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="mobile",NAME="Mobile",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=280474,CODECS="avc1.42C00D,mp4a.40.2",RESOLUTION="400x226",VIDEO="mobile"
https://vod.edgecast.hls.ttvnw.net/e5da31ab49_riotgames_15001215120_261543898/mobile/index-muted-HM49I092CC.m3u8
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="audio_only",NAME="Audio Only",AUTOSELECT=NO,DEFAULT=NO
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=182725,CODECS="mp4a.40.2",VIDEO="audio_only"
https://vod.edgecast.hls.ttvnw.net/e5da31ab49_riotgames_15001215120_261543898/audio_only/index-muted-HM49I092CC.m3u8

10
test/testdata/m3u8/vidio.m3u8 vendored Normal file
View File

@ -0,0 +1,10 @@
#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=300000,RESOLUTION=480x270,NAME="270p 3G"
https://cdn1-a.production.vidio.static6.com/uploads/165683/dj_ambred-4383-b300.mp4.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=600000,RESOLUTION=640x360,NAME="360p SD"
https://cdn1-a.production.vidio.static6.com/uploads/165683/dj_ambred-4383-b600.mp4.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1200000,RESOLUTION=1280x720,NAME="720p HD"
https://cdn1-a.production.vidio.static6.com/uploads/165683/dj_ambred-4383-b1200.mp4.m3u8

18
test/testdata/mpd/float_duration.mpd vendored Normal file
View File

@ -0,0 +1,18 @@
<?xml version="1.0" encoding="UTF-8"?>
<MPD xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:mpeg:dash:schema:mpd:2011" type="static" minBufferTime="PT2S" profiles="urn:mpeg:dash:profile:isoff-on-demand:2011" mediaPresentationDuration="PT6014S">
<Period bitstreamSwitching="true">
<AdaptationSet mimeType="audio/mp4" codecs="mp4a.40.2" startWithSAP="1" segmentAlignment="true">
<SegmentTemplate timescale="1000000" presentationTimeOffset="0" initialization="ai_$RepresentationID$.mp4d" media="a_$RepresentationID$_$Number$.mp4d" duration="2000000.0" startNumber="0"></SegmentTemplate>
<Representation id="318597" bandwidth="61587"></Representation>
</AdaptationSet>
<AdaptationSet mimeType="video/mp4" startWithSAP="1" segmentAlignment="true">
<SegmentTemplate timescale="1000000" presentationTimeOffset="0" initialization="vi_$RepresentationID$.mp4d" media="v_$RepresentationID$_$Number$.mp4d" duration="2000000.0" startNumber="0"></SegmentTemplate>
<Representation id="318597" codecs="avc1.42001f" width="340" height="192" bandwidth="318597"></Representation>
<Representation id="638590" codecs="avc1.42001f" width="512" height="288" bandwidth="638590"></Representation>
<Representation id="1022565" codecs="avc1.4d001f" width="688" height="384" bandwidth="1022565"></Representation>
<Representation id="2046506" codecs="avc1.4d001f" width="1024" height="576" bandwidth="2046506"></Representation>
<Representation id="3998017" codecs="avc1.640029" width="1280" height="720" bandwidth="3998017"></Representation>
<Representation id="5997485" codecs="avc1.640032" width="1920" height="1080" bandwidth="5997485"></Representation>
</AdaptationSet>
</Period>
</MPD>

28
test/testdata/mpd/unfragmented.mpd vendored Normal file
View File

@ -0,0 +1,28 @@
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<MPD mediaPresentationDuration="PT54.915S" minBufferTime="PT1.500S" profiles="urn:mpeg:dash:profile:isoff-on-demand:2011" type="static" xmlns="urn:mpeg:dash:schema:mpd:2011">
<Period duration="PT54.915S">
<AdaptationSet segmentAlignment="true" subsegmentAlignment="true" subsegmentStartsWithSAP="1">
<Representation bandwidth="804261" codecs="avc1.4d401e" frameRate="30" height="360" id="VIDEO-1" mimeType="video/mp4" startWithSAP="1" width="360">
<BaseURL>DASH_360</BaseURL>
<SegmentBase indexRange="915-1114" indexRangeExact="true">
<Initialization range="0-914"/>
</SegmentBase>
</Representation>
<Representation bandwidth="608000" codecs="avc1.4d401e" frameRate="30" height="240" id="VIDEO-2" mimeType="video/mp4" startWithSAP="1" width="240">
<BaseURL>DASH_240</BaseURL>
<SegmentBase indexRange="913-1112" indexRangeExact="true">
<Initialization range="0-912"/>
</SegmentBase>
</Representation>
</AdaptationSet>
<AdaptationSet>
<Representation audioSamplingRate="48000" bandwidth="129870" codecs="mp4a.40.2" id="AUDIO-1" mimeType="audio/mp4" startWithSAP="1">
<AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
<BaseURL>audio</BaseURL>
<SegmentBase indexRange="832-1007" indexRangeExact="true">
<Initialization range="0-831"/>
</SegmentBase>
</Representation>
</AdaptationSet>
</Period>
</MPD>

218
test/testdata/mpd/urls_only.mpd vendored Normal file
View File

@ -0,0 +1,218 @@
<?xml version="1.0" ?>
<MPD maxSegmentDuration="PT0H0M10.000S" mediaPresentationDuration="PT0H4M1.728S" minBufferTime="PT1.500S" profiles="urn:mpeg:dash:profile:isoff-main:2011" type="static" xmlns="urn:mpeg:dash:schema:mpd:2011">
<Period duration="PT0H4M1.728S">
<AdaptationSet bitstreamSwitching="true" lang="und" maxHeight="1080" maxWidth="1920" par="16:9" segmentAlignment="true">
<ContentComponent contentType="video" id="1"/>
<Representation audioSamplingRate="44100" bandwidth="200000" codecs="avc3.42c01e,mp4a.40.2" frameRate="25" height="144" id="h264_aac_144p_m4s" mimeType="video/mp4" sar="1:1" startWithSAP="1" width="256">
<SegmentList duration="10000" timescale="1000">
<Initialization sourceURL="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/init/432f65a0.mp4"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/0/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/1/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/2/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/3/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/4/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/5/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/6/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/7/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/8/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/9/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/10/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/11/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/12/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/13/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/14/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/15/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/16/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/17/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/18/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/19/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/20/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/21/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/22/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/23/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/24/432f65a0.m4s"/>
</SegmentList>
</Representation>
<Representation audioSamplingRate="44100" bandwidth="400000" codecs="avc3.42c01e,mp4a.40.2" frameRate="25" height="240" id="h264_aac_240p_m4s" mimeType="video/mp4" sar="160:159" startWithSAP="1" width="424">
<SegmentList duration="10000" timescale="1000">
<Initialization sourceURL="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/init/432f65a0.mp4"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/0/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/1/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/2/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/3/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/4/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/5/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/6/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/7/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/8/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/9/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/10/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/11/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/12/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/13/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/14/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/15/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/16/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/17/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/18/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/19/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/20/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/21/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/22/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/23/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/24/432f65a0.m4s"/>
</SegmentList>
</Representation>
<Representation audioSamplingRate="44100" bandwidth="800000" codecs="avc3.42c01e,mp4a.40.2" frameRate="25" height="360" id="h264_aac_360p_m4s" mimeType="video/mp4" sar="1:1" startWithSAP="1" width="640">
<SegmentList duration="10000" timescale="1000">
<Initialization sourceURL="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/init/432f65a0.mp4"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/0/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/1/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/2/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/3/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/4/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/5/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/6/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/7/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/8/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/9/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/10/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/11/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/12/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/13/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/14/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/15/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/16/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/17/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/18/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/19/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/20/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/21/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/22/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/23/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/24/432f65a0.m4s"/>
</SegmentList>
</Representation>
<Representation audioSamplingRate="44100" bandwidth="1200000" codecs="avc3.42c01e,mp4a.40.2" frameRate="25" height="480" id="h264_aac_480p_m4s" mimeType="video/mp4" sar="320:321" startWithSAP="1" width="856">
<SegmentList duration="10000" timescale="1000">
<Initialization sourceURL="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/init/432f65a0.mp4"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/0/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/1/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/2/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/3/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/4/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/5/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/6/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/7/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/8/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/9/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/10/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/11/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/12/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/13/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/14/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/15/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/16/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/17/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/18/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/19/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/20/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/21/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/22/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/23/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/24/432f65a0.m4s"/>
</SegmentList>
</Representation>
<Representation audioSamplingRate="44100" bandwidth="1600000" codecs="avc3.42c01e,mp4a.40.2" frameRate="25" height="576" id="h264_aac_576p_m4s" mimeType="video/mp4" sar="1:1" startWithSAP="1" width="1024">
<SegmentList duration="10000" timescale="1000">
<Initialization sourceURL="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/init/432f65a0.mp4"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/0/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/1/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/2/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/3/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/4/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/5/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/6/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/7/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/8/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/9/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/10/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/11/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/12/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/13/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/14/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/15/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/16/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/17/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/18/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/19/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/20/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/21/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/22/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/23/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/24/432f65a0.m4s"/>
</SegmentList>
</Representation>
<Representation audioSamplingRate="44100" bandwidth="2400000" codecs="avc3.42c01e,mp4a.40.2" frameRate="25" height="720" id="h264_aac_720p_m4s" mimeType="video/mp4" sar="1:1" startWithSAP="1" width="1280">
<SegmentList duration="10000" timescale="1000">
<Initialization sourceURL="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/init/432f65a0.mp4"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/0/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/1/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/2/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/3/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/4/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/5/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/6/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/7/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/8/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/9/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/10/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/11/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/12/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/13/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/14/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/15/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/16/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/17/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/18/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/19/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/20/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/21/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/22/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/23/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/24/432f65a0.m4s"/>
</SegmentList>
</Representation>
<Representation audioSamplingRate="44100" bandwidth="4400000" codecs="avc3.42c01e,mp4a.40.2" frameRate="25" height="1080" id="h264_aac_1080p_m4s" mimeType="video/mp4" sar="1:1" startWithSAP="1" width="1920">
<SegmentList duration="10000" timescale="1000">
<Initialization sourceURL="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/init/432f65a0.mp4"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/0/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/1/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/2/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/3/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/4/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/5/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/6/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/7/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/8/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/9/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/10/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/11/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/12/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/13/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/14/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/15/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/16/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/17/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/18/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/19/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/20/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/21/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/22/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/23/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/24/432f65a0.m4s"/>
</SegmentList>
</Representation>
</AdaptationSet>
</Period>
</MPD>

34
test/testdata/xspf/foo_xspf.xspf vendored Normal file
View File

@ -0,0 +1,34 @@
<?xml version="1.0" encoding="UTF-8"?>
<playlist version="1" xmlns="http://xspf.org/ns/0/">
<date>2018-03-09T18:01:43Z</date>
<trackList>
<track>
<location>cd1/track%201.mp3</location>
<title>Pandemonium</title>
<creator>Foilverb</creator>
<annotation>Visit http://bigbrother404.bandcamp.com</annotation>
<album>Pandemonium EP</album>
<trackNum>1</trackNum>
<duration>202416</duration>
</track>
<track>
<location>../%E3%83%88%E3%83%A9%E3%83%83%E3%82%AF%E3%80%80%EF%BC%92.mp3</location>
<title>Final Cartridge (Nichico Twelve Remix)</title>
<annotation>Visit http://bigbrother404.bandcamp.com</annotation>
<creator>Foilverb</creator>
<album>Pandemonium EP</album>
<trackNum>2</trackNum>
<duration>255857</duration>
</track>
<track>
<location>track3.mp3</location>
<location>https://example.com/track3.mp3</location>
<title>Rebuilding Nightingale</title>
<annotation>Visit http://bigbrother404.bandcamp.com</annotation>
<creator>Foilverb</creator>
<album>Pandemonium EP</album>
<trackNum>3</trackNum>
<duration>287915</duration>
</track>
</trackList>
</playlist>

View File

@ -7,7 +7,7 @@
# https://github.com/zsh-users/antigen
# Install youtube-dl:
# antigen bundle rg3/youtube-dl
# antigen bundle ytdl-org/youtube-dl
# Bundles installed by antigen are available for use immediately.
# Update youtube-dl (and all other antigen bundles):

File diff suppressed because it is too large Load Diff

View File

@ -16,7 +16,6 @@ from .options import (
parseOpts,
)
from .compat import (
compat_expanduser,
compat_getpass,
compat_shlex_split,
workaround_optparse_bug9161,
@ -26,6 +25,7 @@ from .utils import (
decodeOption,
DEFAULT_OUTTMPL,
DownloadError,
expand_path,
match_filter_func,
MaxDownloadsReached,
preferredencoding,
@ -48,7 +48,7 @@ from .YoutubeDL import YoutubeDL
def _real_main(argv=None):
# Compatibility fixes for Windows
if sys.platform == 'win32':
# https://github.com/rg3/youtube-dl/issues/820
# https://github.com/ytdl-org/youtube-dl/issues/820
codecs.register(lambda name: codecs.lookup('utf-8') if name == 'cp65001' else None)
workaround_optparse_bug9161()
@ -88,13 +88,13 @@ def _real_main(argv=None):
batchfd = sys.stdin
else:
batchfd = io.open(
compat_expanduser(opts.batchfile),
expand_path(opts.batchfile),
'r', encoding='utf-8', errors='ignore')
batch_urls = read_batch_urls(batchfd)
if opts.verbose:
write_string('[debug] Batch file urls: ' + repr(batch_urls) + '\n')
except IOError:
sys.exit('ERROR: batch file could not be read')
sys.exit('ERROR: batch file %s could not be read' % opts.batchfile)
all_urls = batch_urls + [url.strip() for url in args] # batch_urls are already striped in read_batch_urls
_enc = preferredencoding()
all_urls = [url.decode(_enc, 'ignore') if isinstance(url, bytes) else url for url in all_urls]
@ -166,6 +166,8 @@ def _real_main(argv=None):
if opts.max_sleep_interval is not None:
if opts.max_sleep_interval < 0:
parser.error('max sleep interval must be positive or 0')
if opts.sleep_interval is None:
parser.error('min sleep interval must be specified, use --min-sleep-interval')
if opts.max_sleep_interval < opts.sleep_interval:
parser.error('max sleep interval must be greater than or equal to min sleep interval')
else:
@ -191,12 +193,17 @@ def _real_main(argv=None):
if numeric_buffersize is None:
parser.error('invalid buffer size specified')
opts.buffersize = numeric_buffersize
if opts.http_chunk_size is not None:
numeric_chunksize = FileDownloader.parse_bytes(opts.http_chunk_size)
if not numeric_chunksize:
parser.error('invalid http chunk size specified')
opts.http_chunk_size = numeric_chunksize
if opts.playliststart <= 0:
raise ValueError('Playlist start must be positive')
if opts.playlistend not in (-1, None) and opts.playlistend < opts.playliststart:
raise ValueError('Playlist end must be greater than playlist start')
if opts.extractaudio:
if opts.audioformat not in ['best', 'aac', 'mp3', 'm4a', 'opus', 'vorbis', 'wav']:
if opts.audioformat not in ['best', 'aac', 'flac', 'mp3', 'm4a', 'opus', 'vorbis', 'wav']:
parser.error('invalid audio format specified')
if opts.audioquality:
opts.audioquality = opts.audioquality.strip('k').strip('K')
@ -206,7 +213,7 @@ def _real_main(argv=None):
if opts.recodevideo not in ['mp4', 'flv', 'webm', 'ogg', 'mkv', 'avi']:
parser.error('invalid video recode format specified')
if opts.convertsubtitles is not None:
if opts.convertsubtitles not in ['srt', 'vtt', 'ass']:
if opts.convertsubtitles not in ['srt', 'vtt', 'ass', 'lrc']:
parser.error('invalid subtitle format specified')
if opts.date is not None:
@ -223,14 +230,14 @@ def _real_main(argv=None):
if opts.allsubtitles and not opts.writeautomaticsub:
opts.writesubtitles = True
outtmpl = ((opts.outtmpl is not None and opts.outtmpl) or
(opts.format == '-1' and opts.usetitle and '%(title)s-%(id)s-%(format)s.%(ext)s') or
(opts.format == '-1' and '%(id)s-%(format)s.%(ext)s') or
(opts.usetitle and opts.autonumber and '%(autonumber)s-%(title)s-%(id)s.%(ext)s') or
(opts.usetitle and '%(title)s-%(id)s.%(ext)s') or
(opts.useid and '%(id)s.%(ext)s') or
(opts.autonumber and '%(autonumber)s-%(id)s.%(ext)s') or
DEFAULT_OUTTMPL)
outtmpl = ((opts.outtmpl is not None and opts.outtmpl)
or (opts.format == '-1' and opts.usetitle and '%(title)s-%(id)s-%(format)s.%(ext)s')
or (opts.format == '-1' and '%(id)s-%(format)s.%(ext)s')
or (opts.usetitle and opts.autonumber and '%(autonumber)s-%(title)s-%(id)s.%(ext)s')
or (opts.usetitle and '%(title)s-%(id)s.%(ext)s')
or (opts.useid and '%(id)s.%(ext)s')
or (opts.autonumber and '%(autonumber)s-%(id)s.%(ext)s')
or DEFAULT_OUTTMPL)
if not os.path.splitext(outtmpl)[1] and opts.extractaudio:
parser.error('Cannot download a video and extract audio into the same'
' file! Use "{0}.%(ext)s" instead of "{0}" as the output'
@ -238,18 +245,15 @@ def _real_main(argv=None):
any_getting = opts.geturl or opts.gettitle or opts.getid or opts.getthumbnail or opts.getdescription or opts.getfilename or opts.getformat or opts.getduration or opts.dumpjson or opts.dump_single_json
any_printing = opts.print_json
download_archive_fn = compat_expanduser(opts.download_archive) if opts.download_archive is not None else opts.download_archive
download_archive_fn = expand_path(opts.download_archive) if opts.download_archive is not None else opts.download_archive
# PostProcessors
postprocessors = []
# Add the metadata pp first, the other pps will copy it
if opts.metafromtitle:
postprocessors.append({
'key': 'MetadataFromTitle',
'titleformat': opts.metafromtitle
})
if opts.addmetadata:
postprocessors.append({'key': 'FFmpegMetadata'})
if opts.extractaudio:
postprocessors.append({
'key': 'FFmpegExtractAudio',
@ -262,6 +266,16 @@ def _real_main(argv=None):
'key': 'FFmpegVideoConvertor',
'preferedformat': opts.recodevideo,
})
# FFmpegMetadataPP should be run after FFmpegVideoConvertorPP and
# FFmpegExtractAudioPP as containers before conversion may not support
# metadata (3gp, webm, etc.)
# And this post-processor should be placed before other metadata
# manipulating post-processors (FFmpegEmbedSubtitle) to prevent loss of
# extra metadata. By default ffmpeg preserves metadata applicable for both
# source and target containers. From this point the container won't change,
# so metadata can be added here.
if opts.addmetadata:
postprocessors.append({'key': 'FFmpegMetadata'})
if opts.convertsubtitles:
postprocessors.append({
'key': 'FFmpegSubtitlesConvertor',
@ -336,8 +350,10 @@ def _real_main(argv=None):
'retries': opts.retries,
'fragment_retries': opts.fragment_retries,
'skip_unavailable_fragments': opts.skip_unavailable_fragments,
'keep_fragments': opts.keep_fragments,
'buffersize': opts.buffersize,
'noresizebuffer': opts.noresizebuffer,
'http_chunk_size': opts.http_chunk_size,
'continuedl': opts.continue_dl,
'noprogress': opts.noprogress,
'progress_with_newline': opts.progress_with_newline,
@ -414,6 +430,12 @@ def _real_main(argv=None):
'cn_verification_proxy': opts.cn_verification_proxy,
'geo_verification_proxy': opts.geo_verification_proxy,
'config_location': opts.config_location,
'geo_bypass': opts.geo_bypass,
'geo_bypass_country': opts.geo_bypass_country,
'geo_bypass_ip_block': opts.geo_bypass_ip_block,
# just for deprecation check
'autonumber': opts.autonumber if opts.autonumber is True else None,
'usetitle': opts.usetitle if opts.usetitle is True else None,
}
with YoutubeDL(ydl_opts) as ydl:
@ -437,7 +459,7 @@ def _real_main(argv=None):
try:
if opts.load_info_filename is not None:
retcode = ydl.download_with_info_file(compat_expanduser(opts.load_info_filename))
retcode = ydl.download_with_info_file(expand_path(opts.load_info_filename))
else:
retcode = ydl.download(all_urls)
except MaxDownloadsReached:

View File

@ -1,8 +1,8 @@
from __future__ import unicode_literals
import base64
from math import ceil
from .compat import compat_b64decode
from .utils import bytes_to_intlist, intlist_to_bytes
BLOCK_SIZE_BYTES = 16
@ -60,6 +60,34 @@ def aes_cbc_decrypt(data, key, iv):
return decrypted_data
def aes_cbc_encrypt(data, key, iv):
"""
Encrypt with aes in CBC mode. Using PKCS#7 padding
@param {int[]} data cleartext
@param {int[]} key 16/24/32-Byte cipher key
@param {int[]} iv 16-Byte IV
@returns {int[]} encrypted data
"""
expanded_key = key_expansion(key)
block_count = int(ceil(float(len(data)) / BLOCK_SIZE_BYTES))
encrypted_data = []
previous_cipher_block = iv
for i in range(block_count):
block = data[i * BLOCK_SIZE_BYTES: (i + 1) * BLOCK_SIZE_BYTES]
remaining_length = BLOCK_SIZE_BYTES - len(block)
block += [remaining_length] * remaining_length
mixed_block = xor(block, previous_cipher_block)
encrypted_block = aes_encrypt(mixed_block, expanded_key)
encrypted_data += encrypted_block
previous_cipher_block = encrypted_block
return encrypted_data
def key_expansion(data):
"""
Generate key schedule
@ -152,7 +180,7 @@ def aes_decrypt_text(data, password, key_size_bytes):
"""
NONCE_LENGTH_BYTES = 8
data = bytes_to_intlist(base64.b64decode(data.encode('utf-8')))
data = bytes_to_intlist(compat_b64decode(data))
password = bytes_to_intlist(password.encode('utf-8'))
key = password[:key_size_bytes] + [0] * (key_size_bytes - len(password))

View File

@ -8,8 +8,11 @@ import re
import shutil
import traceback
from .compat import compat_expanduser, compat_getenv
from .utils import write_json_file
from .compat import compat_getenv
from .utils import (
expand_path,
write_json_file,
)
class Cache(object):
@ -21,7 +24,7 @@ class Cache(object):
if res is None:
cache_root = compat_getenv('XDG_CACHE_HOME', '~/.cache')
res = os.path.join(cache_root, 'youtube-dl')
return compat_expanduser(res)
return expand_path(res)
def _get_cache_fn(self, section, key, dtype):
assert re.match(r'^[a-zA-Z0-9_.-]+$', section), \

View File

@ -1,13 +1,17 @@
# coding: utf-8
from __future__ import unicode_literals
import base64
import binascii
import collections
import ctypes
import email
import getpass
import io
import itertools
import optparse
import os
import platform
import re
import shlex
import shutil
@ -15,7 +19,6 @@ import socket
import struct
import subprocess
import sys
import itertools
import xml.etree.ElementTree
@ -54,6 +57,17 @@ try:
except ImportError: # Python 2
import cookielib as compat_cookiejar
if sys.version_info[0] == 2:
class compat_cookiejar_Cookie(compat_cookiejar.Cookie):
def __init__(self, version, name, value, *args, **kwargs):
if isinstance(name, compat_str):
name = name.encode()
if isinstance(value, compat_str):
value = value.encode()
compat_cookiejar.Cookie.__init__(self, version, name, value, *args, **kwargs)
else:
compat_cookiejar_Cookie = compat_cookiejar.Cookie
try:
import http.cookies as compat_cookies
except ImportError: # Python 2
@ -2322,6 +2336,19 @@ try:
except ImportError: # Python 2
from HTMLParser import HTMLParser as compat_HTMLParser
try: # Python 2
from HTMLParser import HTMLParseError as compat_HTMLParseError
except ImportError: # Python <3.4
try:
from html.parser import HTMLParseError as compat_HTMLParseError
except ImportError: # Python >3.4
# HTMLParseError has been deprecated in Python 3.3 and removed in
# Python 3.5. Introducing dummy exception for Python >3.5 for compatible
# and uniform cross-version exception handling
class compat_HTMLParseError(Exception):
pass
try:
from subprocess import DEVNULL
compat_subprocess_get_DEVNULL = lambda: DEVNULL
@ -2348,7 +2375,7 @@ except ImportError: # Python 2
# HACK: The following are the correct unquote_to_bytes, unquote and unquote_plus
# implementations from cpython 3.4.3's stdlib. Python 2's version
# is apparently broken (see https://github.com/rg3/youtube-dl/pull/6244)
# is apparently broken (see https://github.com/ytdl-org/youtube-dl/pull/6244)
def compat_urllib_parse_unquote_to_bytes(string):
"""unquote_to_bytes('abc%20def') -> b'abc def'."""
@ -2492,6 +2519,15 @@ class _TreeBuilder(etree.TreeBuilder):
pass
try:
# xml.etree.ElementTree.Element is a method in Python <=2.6 and
# the following will crash with:
# TypeError: isinstance() arg 2 must be a class, type, or tuple of classes and types
isinstance(None, xml.etree.ElementTree.Element)
from xml.etree.ElementTree import Element as compat_etree_Element
except TypeError: # Python <=2.6
from xml.etree.ElementTree import _ElementInterface as compat_etree_Element
if sys.version_info[0] >= 3:
def compat_etree_fromstring(text):
return etree.XML(text, parser=etree.XMLParser(target=_TreeBuilder()))
@ -2604,6 +2640,14 @@ except ImportError: # Python 2
parsed_result[name] = [value]
return parsed_result
compat_os_name = os._name if os.name == 'java' else os.name
if compat_os_name == 'nt':
def compat_shlex_quote(s):
return s if re.match(r'^[-_\w./]+$', s) else '"%s"' % s.replace('"', '\\"')
else:
try:
from shlex import quote as compat_shlex_quote
except ImportError: # Python < 3.3
@ -2616,9 +2660,9 @@ except ImportError: # Python < 3.3
try:
args = shlex.split('中文')
assert (isinstance(args, list) and
isinstance(args[0], compat_str) and
args[0] == '中文')
assert (isinstance(args, list)
and isinstance(args[0], compat_str)
and args[0] == '中文')
compat_shlex_split = shlex.split
except (AssertionError, UnicodeEncodeError):
# Working around shlex issue with unicode strings on some python 2
@ -2636,9 +2680,6 @@ def compat_ord(c):
return ord(c)
compat_os_name = os._name if os.name == 'java' else os.name
if sys.version_info >= (3, 0):
compat_getenv = os.getenv
compat_expanduser = os.path.expanduser
@ -2692,7 +2733,7 @@ else:
userhome = pwent.pw_dir
userhome = userhome.rstrip('/')
return (userhome + path[i:]) or '/'
elif compat_os_name == 'nt' or compat_os_name == 'ce':
elif compat_os_name in ('nt', 'ce'):
def compat_expanduser(path):
"""Expand ~ and ~user constructs.
@ -2724,6 +2765,17 @@ else:
compat_expanduser = os.path.expanduser
if compat_os_name == 'nt' and sys.version_info < (3, 8):
# os.path.realpath on Windows does not follow symbolic links
# prior to Python 3.8 (see https://bugs.python.org/issue9949)
def compat_realpath(path):
while os.path.islink(path):
path = os.path.abspath(os.readlink(path))
return path
else:
compat_realpath = os.path.realpath
if sys.version_info < (3, 0):
def compat_print(s):
from .utils import preferredencoding
@ -2760,6 +2812,18 @@ else:
compat_kwargs = lambda kwargs: kwargs
try:
compat_numeric_types = (int, float, long, complex)
except NameError: # Python 3
compat_numeric_types = (int, float, complex)
try:
compat_integer_types = (int, long)
except NameError: # Python 3
compat_integer_types = (int, )
if sys.version_info < (2, 7):
def compat_socket_create_connection(address, timeout, source_address=None):
host, port = address
@ -2786,7 +2850,7 @@ else:
compat_socket_create_connection = socket.create_connection
# Fix https://github.com/rg3/youtube-dl/issues/4223
# Fix https://github.com/ytdl-org/youtube-dl/issues/4223
# See http://bugs.python.org/issue9161 for what is broken
def workaround_optparse_bug9161():
op = optparse.OptionParser()
@ -2870,18 +2934,74 @@ except TypeError:
if isinstance(spec, compat_str):
spec = spec.encode('ascii')
return struct.unpack(spec, *args)
class compat_Struct(struct.Struct):
def __init__(self, fmt):
if isinstance(fmt, compat_str):
fmt = fmt.encode('ascii')
super(compat_Struct, self).__init__(fmt)
else:
compat_struct_pack = struct.pack
compat_struct_unpack = struct.unpack
if platform.python_implementation() == 'IronPython' and sys.version_info < (2, 7, 8):
class compat_Struct(struct.Struct):
def unpack(self, string):
if not isinstance(string, buffer): # noqa: F821
string = buffer(string) # noqa: F821
return super(compat_Struct, self).unpack(string)
else:
compat_Struct = struct.Struct
try:
from future_builtins import zip as compat_zip
except ImportError: # not 2.6+ or is 3.x
try:
from itertools import izip as compat_zip # < 2.5 or 3.x
except ImportError:
compat_zip = zip
if sys.version_info < (3, 3):
def compat_b64decode(s, *args, **kwargs):
if isinstance(s, compat_str):
s = s.encode('ascii')
return base64.b64decode(s, *args, **kwargs)
else:
compat_b64decode = base64.b64decode
if platform.python_implementation() == 'PyPy' and sys.pypy_version_info < (5, 4, 0):
# PyPy2 prior to version 5.4.0 expects byte strings as Windows function
# names, see the original PyPy issue [1] and the youtube-dl one [2].
# 1. https://bitbucket.org/pypy/pypy/issues/2360/windows-ctypescdll-typeerror-function-name
# 2. https://github.com/ytdl-org/youtube-dl/pull/4392
def compat_ctypes_WINFUNCTYPE(*args, **kwargs):
real = ctypes.WINFUNCTYPE(*args, **kwargs)
def resf(tpl, *args, **kwargs):
funcname, dll = tpl
return real((str(funcname), dll), *args, **kwargs)
return resf
else:
def compat_ctypes_WINFUNCTYPE(*args, **kwargs):
return ctypes.WINFUNCTYPE(*args, **kwargs)
__all__ = [
'compat_HTMLParseError',
'compat_HTMLParser',
'compat_HTTPError',
'compat_Struct',
'compat_b64decode',
'compat_basestring',
'compat_chr',
'compat_cookiejar',
'compat_cookiejar_Cookie',
'compat_cookies',
'compat_ctypes_WINFUNCTYPE',
'compat_etree_Element',
'compat_etree_fromstring',
'compat_etree_register_namespace',
'compat_expanduser',
@ -2893,12 +3013,15 @@ __all__ = [
'compat_http_client',
'compat_http_server',
'compat_input',
'compat_integer_types',
'compat_itertools_count',
'compat_kwargs',
'compat_numeric_types',
'compat_ord',
'compat_os_name',
'compat_parse_qs',
'compat_print',
'compat_realpath',
'compat_setenv',
'compat_shlex_quote',
'compat_shlex_split',
@ -2922,5 +3045,6 @@ __all__ = [
'compat_urlretrieve',
'compat_xml_parse_error',
'compat_xpath',
'compat_zip',
'workaround_optparse_bug9161',
]

View File

@ -43,6 +43,9 @@ def get_suitable_downloader(info_dict, params={}):
if ed.can_download(info_dict):
return ed
if protocol.startswith('m3u8') and info_dict.get('is_live'):
return FFmpegFD
if protocol == 'm3u8' and params.get('hls_prefer_native') is True:
return HlsFD

View File

@ -8,10 +8,11 @@ import random
from ..compat import compat_os_name
from ..utils import (
decodeArgument,
encodeFilename,
error_to_compat_str,
decodeArgument,
format_bytes,
shell_quote,
timeconvert,
)
@ -44,10 +45,12 @@ class FileDownloader(object):
min_filesize: Skip files smaller than this size
max_filesize: Skip files larger than this size
xattr_set_filesize: Set ytdl.filesize user xattribute with expected size.
(experimental)
external_downloader_args: A list of additional command-line arguments for the
external downloader.
hls_use_mpegts: Use the mpegts container for HLS videos.
http_chunk_size: Size of a chunk for chunk-based HTTP downloading. May be
useful for bypassing bandwidth throttling imposed by
a webserver (experimental)
Subclasses of this one must re-define the real_download method.
"""
@ -173,7 +176,9 @@ class FileDownloader(object):
return
speed = float(byte_counter) / elapsed
if speed > rate_limit:
time.sleep(max((byte_counter // rate_limit) - elapsed, 0))
sleep_time = float(byte_counter) / rate_limit - elapsed
if sleep_time > 0:
time.sleep(sleep_time)
def temp_name(self, filename):
"""Returns a temporary filename for the given filename."""
@ -187,6 +192,9 @@ class FileDownloader(object):
return filename[:-len('.part')]
return filename
def ytdl_filename(self, filename):
return filename + '.ytdl'
def try_rename(self, old_filename, new_filename):
try:
if old_filename == new_filename:
@ -242,12 +250,13 @@ class FileDownloader(object):
if self.params.get('noprogress', False):
self.to_screen('[download] Download completed')
else:
msg_template = '100%%'
if s.get('total_bytes') is not None:
s['_total_bytes_str'] = format_bytes(s['total_bytes'])
msg_template += ' of %(_total_bytes_str)s'
if s.get('elapsed') is not None:
s['_elapsed_str'] = self.format_seconds(s['elapsed'])
msg_template = '100%% of %(_total_bytes_str)s in %(_elapsed_str)s'
else:
msg_template = '100%% of %(_total_bytes_str)s'
msg_template += ' in %(_elapsed_str)s'
self._report_progress_status(
msg_template % s, is_last_line=True)
@ -300,11 +309,11 @@ class FileDownloader(object):
"""Report attempt to resume at given byte."""
self.to_screen('[download] Resuming download at byte %s' % resume_len)
def report_retry(self, count, retries):
def report_retry(self, err, count, retries):
"""Report retry in case of HTTP error 5xx"""
self.to_screen(
'[download] Got server HTTP error. Retrying (attempt %d of %s)...'
% (count, self.format_retries(retries)))
'[download] Got server HTTP error: %s. Retrying (attempt %d of %s)...'
% (error_to_compat_str(err), count, self.format_retries(retries)))
def report_file_already_downloaded(self, file_name):
"""Report file has already been fully downloaded."""
@ -323,14 +332,15 @@ class FileDownloader(object):
"""
nooverwrites_and_exists = (
self.params.get('nooverwrites', False) and
os.path.exists(encodeFilename(filename))
self.params.get('nooverwrites', False)
and os.path.exists(encodeFilename(filename))
)
if not hasattr(filename, 'write'):
continuedl_and_exists = (
self.params.get('continuedl', True) and
os.path.isfile(encodeFilename(filename)) and
not self.params.get('nopart', False)
self.params.get('continuedl', True)
and os.path.isfile(encodeFilename(filename))
and not self.params.get('nopart', False)
)
# Check file already present
@ -347,7 +357,10 @@ class FileDownloader(object):
if min_sleep_interval:
max_sleep_interval = self.params.get('max_sleep_interval', min_sleep_interval)
sleep_interval = random.uniform(min_sleep_interval, max_sleep_interval)
self.to_screen('[download] Sleeping %s seconds...' % sleep_interval)
self.to_screen(
'[download] Sleeping %s seconds...' % (
int(sleep_interval) if sleep_interval.is_integer()
else '%.2f' % sleep_interval))
time.sleep(sleep_interval)
return self.real_download(filename, info_dict)
@ -374,10 +387,5 @@ class FileDownloader(object):
if exe is None:
exe = os.path.basename(str_args[0])
try:
import pipes
shell_quote = lambda args: ' '.join(map(pipes.quote, str_args))
except ImportError:
shell_quote = repr
self.to_screen('[debug] %s command line: %s' % (
exe, shell_quote(str_args)))

View File

@ -1,12 +1,10 @@
from __future__ import unicode_literals
import os
from .fragment import FragmentFD
from ..compat import compat_urllib_error
from ..utils import (
sanitize_open,
encodeFilename,
DownloadError,
urljoin,
)
@ -18,64 +16,65 @@ class DashSegmentsFD(FragmentFD):
FD_NAME = 'dashsegments'
def real_download(self, filename, info_dict):
segments = info_dict['fragments'][:1] if self.params.get(
fragment_base_url = info_dict.get('fragment_base_url')
fragments = info_dict['fragments'][:1] if self.params.get(
'test', False) else info_dict['fragments']
ctx = {
'filename': filename,
'total_frags': len(segments),
'total_frags': len(fragments),
}
self._prepare_and_start_frag_download(ctx)
segments_filenames = []
fragment_retries = self.params.get('fragment_retries', 0)
skip_unavailable_fragments = self.params.get('skip_unavailable_fragments', True)
def process_segment(segment, tmp_filename, num):
segment_url = segment['url']
segment_name = 'Frag%d' % num
target_filename = '%s-%s' % (tmp_filename, segment_name)
frag_index = 0
for i, fragment in enumerate(fragments):
frag_index += 1
if frag_index <= ctx['fragment_index']:
continue
# In DASH, the first segment contains necessary headers to
# generate a valid MP4 file, so always abort for the first segment
fatal = num == 0 or not skip_unavailable_fragments
fatal = i == 0 or not skip_unavailable_fragments
count = 0
while count <= fragment_retries:
try:
success = ctx['dl'].download(target_filename, {'url': segment_url})
fragment_url = fragment.get('url')
if not fragment_url:
assert fragment_base_url
fragment_url = urljoin(fragment_base_url, fragment['path'])
success, frag_content = self._download_fragment(ctx, fragment_url, info_dict)
if not success:
return False
down, target_sanitized = sanitize_open(target_filename, 'rb')
ctx['dest_stream'].write(down.read())
down.close()
segments_filenames.append(target_sanitized)
self._append_fragment(ctx, frag_content)
break
except compat_urllib_error.HTTPError as err:
# YouTube may often return 404 HTTP error for a fragment causing the
# whole download to fail. However if the same fragment is immediately
# retried with the same request data this usually succeeds (1-2 attemps
# retried with the same request data this usually succeeds (1-2 attempts
# is usually enough) thus allowing to download the whole file successfully.
# To be future-proof we will retry all fragments that fail with any
# HTTP error.
count += 1
if count <= fragment_retries:
self.report_retry_fragment(err, segment_name, count, fragment_retries)
self.report_retry_fragment(err, frag_index, count, fragment_retries)
except DownloadError:
# Don't retry fragment if error occurred during HTTP downloading
# itself since it has own retry settings
if not fatal:
self.report_skip_fragment(frag_index)
break
raise
if count > fragment_retries:
if not fatal:
self.report_skip_fragment(segment_name)
return True
self.report_skip_fragment(frag_index)
continue
self.report_error('giving up after %s fragment retries' % fragment_retries)
return False
return True
for i, segment in enumerate(segments):
if not process_segment(segment, ctx['tmpfilename'], i):
return False
self._finish_frag_download(ctx)
for segment_file in segments_filenames:
os.remove(encodeFilename(segment_file))
return True

View File

@ -1,12 +1,16 @@
from __future__ import unicode_literals
import os.path
import re
import subprocess
import sys
import re
import time
from .common import FileDownloader
from ..compat import compat_setenv
from ..compat import (
compat_setenv,
compat_str,
)
from ..postprocessor.ffmpeg import FFmpegPostProcessor, EXT_TO_OUT_FORMATS
from ..utils import (
cli_option,
@ -26,17 +30,33 @@ class ExternalFD(FileDownloader):
self.report_destination(filename)
tmpfilename = self.temp_name(filename)
try:
started = time.time()
retval = self._call_downloader(tmpfilename, info_dict)
except KeyboardInterrupt:
if not info_dict.get('is_live'):
raise
# Live stream downloading cancellation should be considered as
# correct and expected termination thus all postprocessing
# should take place
retval = 0
self.to_screen('[%s] Interrupted by user' % self.get_basename())
if retval == 0:
status = {
'filename': filename,
'status': 'finished',
'elapsed': time.time() - started,
}
if filename != '-':
fsize = os.path.getsize(encodeFilename(tmpfilename))
self.to_screen('\r[%s] Downloaded %s bytes' % (self.get_basename(), fsize))
self.try_rename(tmpfilename, filename)
self._hook_progress({
status.update({
'downloaded_bytes': fsize,
'total_bytes': fsize,
'filename': filename,
'status': 'finished',
})
self._hook_progress(status)
return True
else:
self.to_stderr('\n')
@ -101,7 +121,11 @@ class CurlFD(ExternalFD):
cmd += self._valueless_option('--silent', 'noprogress')
cmd += self._valueless_option('--verbose', 'verbose')
cmd += self._option('--limit-rate', 'ratelimit')
cmd += self._option('--retry', 'retries')
retry = self._option('--retry', 'retries')
if len(retry) == 2:
if retry[1] in ('inf', 'infinite'):
retry[1] = '2147483647'
cmd += retry
cmd += self._option('--max-filesize', 'max_filesize')
cmd += self._option('--interface', 'source_address')
cmd += self._option('--proxy', 'proxy')
@ -140,6 +164,12 @@ class WgetFD(ExternalFD):
cmd = [self.exe, '-O', tmpfilename, '-nv', '--no-cookies']
for key, val in info_dict['http_headers'].items():
cmd += ['--header', '%s: %s' % (key, val)]
cmd += self._option('--limit-rate', 'ratelimit')
retry = self._option('--tries', 'retries')
if len(retry) == 2:
if retry[1] in ('inf', 'infinite'):
retry[1] = '0'
cmd += retry
cmd += self._option('--bind-address', 'source_address')
cmd += self._option('--proxy', 'proxy')
cmd += self._valueless_option('--no-check-certificate', 'nocheckcertificate')
@ -164,6 +194,7 @@ class Aria2cFD(ExternalFD):
cmd += self._option('--interface', 'source_address')
cmd += self._option('--all-proxy', 'proxy')
cmd += self._bool_option('--check-certificate', 'nocheckcertificate', 'false', 'true', '=')
cmd += self._bool_option('--remote-time', 'updatetime', 'true', 'false', '=')
cmd += ['--', info_dict['url']]
return cmd
@ -199,12 +230,17 @@ class FFmpegFD(ExternalFD):
args = [ffpp.executable, '-y']
for log_level in ('quiet', 'verbose'):
if self.params.get(log_level, False):
args += ['-loglevel', log_level]
break
seekable = info_dict.get('_seekable')
if seekable is not None:
# setting -seekable prevents ffmpeg from guessing if the server
# supports seeking(by adding the header `Range: bytes=0-`), which
# can cause problems in some cases
# https://github.com/rg3/youtube-dl/issues/11800#issuecomment-275037127
# https://github.com/ytdl-org/youtube-dl/issues/11800#issuecomment-275037127
# http://trac.ffmpeg.org/ticket/6125#comment:10
args += ['-seekable', '1' if seekable else '0']
@ -254,6 +290,7 @@ class FFmpegFD(ExternalFD):
tc_url = info_dict.get('tc_url')
flash_version = info_dict.get('flash_version')
live = info_dict.get('rtmp_live', False)
conn = info_dict.get('rtmp_conn')
if player_url is not None:
args += ['-rtmp_swfverify', player_url]
if page_url is not None:
@ -268,8 +305,17 @@ class FFmpegFD(ExternalFD):
args += ['-rtmp_flashver', flash_version]
if live:
args += ['-rtmp_live', 'live']
if isinstance(conn, list):
for entry in conn:
args += ['-rtmp_conn', entry]
elif isinstance(conn, compat_str):
args += ['-rtmp_conn', conn]
args += ['-i', url, '-c', 'copy']
if self.params.get('test', False):
args += ['-fs', compat_str(self._TEST_FILE_SIZE)]
if protocol in ('m3u8', 'm3u8_native'):
if self.params.get('hls_use_mpegts', False) or tmpfilename == '-':
args += ['-f', 'mpegts']
@ -295,7 +341,7 @@ class FFmpegFD(ExternalFD):
# mp4 file couldn't be played, but if we ask ffmpeg to quit it
# produces a file that is playable (this is mostly useful for live
# streams). Note that Windows is not affected and produces playable
# files (see https://github.com/rg3/youtube-dl/issues/8300).
# files (see https://github.com/ytdl-org/youtube-dl/issues/8300).
if sys.platform != 'win32':
proc.communicate(b'q')
raise

View File

@ -1,13 +1,12 @@
from __future__ import division, unicode_literals
import base64
import io
import itertools
import os
import time
from .fragment import FragmentFD
from ..compat import (
compat_b64decode,
compat_etree_fromstring,
compat_urlparse,
compat_urllib_error,
@ -16,9 +15,7 @@ from ..compat import (
compat_struct_unpack,
)
from ..utils import (
encodeFilename,
fix_xml_ampersands,
sanitize_open,
xpath_text,
)
@ -241,13 +238,22 @@ def write_metadata_tag(stream, metadata):
def remove_encrypted_media(media):
return list(filter(lambda e: 'drmAdditionalHeaderId' not in e.attrib and
'drmAdditionalHeaderSetId' not in e.attrib,
return list(filter(lambda e: 'drmAdditionalHeaderId' not in e.attrib
and 'drmAdditionalHeaderSetId' not in e.attrib,
media))
def _add_ns(prop):
return '{http://ns.adobe.com/f4m/1.0}%s' % prop
def _add_ns(prop, ver=1):
return '{http://ns.adobe.com/f4m/%d.0}%s' % (ver, prop)
def get_base_url(manifest):
base_url = xpath_text(
manifest, [_add_ns('baseURL'), _add_ns('baseURL', 2)],
'base URL', default=None)
if base_url:
base_url = base_url.strip()
return base_url
class F4mFD(FragmentFD):
@ -261,8 +267,8 @@ class F4mFD(FragmentFD):
media = doc.findall(_add_ns('media'))
if not media:
self.report_error('No media found')
for e in (doc.findall(_add_ns('drmAdditionalHeader')) +
doc.findall(_add_ns('drmAdditionalHeaderSet'))):
for e in (doc.findall(_add_ns('drmAdditionalHeader'))
+ doc.findall(_add_ns('drmAdditionalHeaderSet'))):
# If id attribute is missing it's valid for all media nodes
# without drmAdditionalHeaderId or drmAdditionalHeaderSetId attribute
if 'id' not in e.attrib:
@ -306,7 +312,7 @@ class F4mFD(FragmentFD):
boot_info = self._get_bootstrap_from_url(bootstrap_url)
else:
bootstrap_url = None
bootstrap = base64.b64decode(node.text.encode('ascii'))
bootstrap = compat_b64decode(node.text)
boot_info = read_bootstrap_info(bootstrap)
return boot_info, bootstrap_url
@ -318,8 +324,8 @@ class F4mFD(FragmentFD):
urlh = self.ydl.urlopen(self._prepare_url(info_dict, man_url))
man_url = urlh.geturl()
# Some manifests may be malformed, e.g. prosiebensat1 generated manifests
# (see https://github.com/rg3/youtube-dl/issues/6215#issuecomment-121704244
# and https://github.com/rg3/youtube-dl/issues/7823)
# (see https://github.com/ytdl-org/youtube-dl/issues/6215#issuecomment-121704244
# and https://github.com/ytdl-org/youtube-dl/issues/7823)
manifest = fix_xml_ampersands(urlh.read().decode('utf-8', 'ignore')).strip()
doc = compat_etree_fromstring(manifest)
@ -333,17 +339,17 @@ class F4mFD(FragmentFD):
rate, media = list(filter(
lambda f: int(f[0]) == requested_bitrate, formats))[0]
base_url = compat_urlparse.urljoin(man_url, media.attrib['url'])
# Prefer baseURL for relative URLs as per 11.2 of F4M 3.0 spec.
man_base_url = get_base_url(doc) or man_url
base_url = compat_urlparse.urljoin(man_base_url, media.attrib['url'])
bootstrap_node = doc.find(_add_ns('bootstrapInfo'))
# From Adobe F4M 3.0 spec:
# The <baseURL> element SHALL be the base URL for all relative
# (HTTP-based) URLs in the manifest. If <baseURL> is not present, said
# URLs should be relative to the location of the containing document.
boot_info, bootstrap_url = self._parse_bootstrap_node(bootstrap_node, man_url)
boot_info, bootstrap_url = self._parse_bootstrap_node(
bootstrap_node, man_base_url)
live = boot_info['live']
metadata_node = media.find(_add_ns('metadata'))
if metadata_node is not None:
metadata = base64.b64decode(metadata_node.text.encode('ascii'))
metadata = compat_b64decode(metadata_node.text)
else:
metadata = None
@ -366,6 +372,7 @@ class F4mFD(FragmentFD):
dest_stream = ctx['dest_stream']
if ctx['complete_frags_downloaded_bytes'] == 0:
write_flv_header(dest_stream)
if not live:
write_metadata_tag(dest_stream, metadata)
@ -374,9 +381,12 @@ class F4mFD(FragmentFD):
self._start_frag_download(ctx)
frags_filenames = []
frag_index = 0
while fragments_list:
seg_i, frag_i = fragments_list.pop(0)
frag_index += 1
if frag_index <= ctx['fragment_index']:
continue
name = 'Seg%d-Frag%d' % (seg_i, frag_i)
query = []
if base_url_parsed.query:
@ -386,17 +396,10 @@ class F4mFD(FragmentFD):
if info_dict.get('extra_param_to_segment_url'):
query.append(info_dict['extra_param_to_segment_url'])
url_parsed = base_url_parsed._replace(path=base_url_parsed.path + name, query='&'.join(query))
frag_filename = '%s-%s' % (ctx['tmpfilename'], name)
try:
success = ctx['dl'].download(frag_filename, {
'url': url_parsed.geturl(),
'http_headers': info_dict.get('http_headers'),
})
success, down_data = self._download_fragment(ctx, url_parsed.geturl(), info_dict)
if not success:
return False
(down, frag_sanitized) = sanitize_open(frag_filename, 'rb')
down_data = down.read()
down.close()
reader = FlvReader(down_data)
while True:
try:
@ -406,17 +409,13 @@ class F4mFD(FragmentFD):
# In tests, segments may be truncated, and thus
# FlvReader may not be able to parse the whole
# chunk. If so, write the segment as is
# See https://github.com/rg3/youtube-dl/issues/9214
# See https://github.com/ytdl-org/youtube-dl/issues/9214
dest_stream.write(down_data)
break
raise
if box_type == b'mdat':
dest_stream.write(box_data)
self._append_fragment(ctx, box_data)
break
if live:
os.remove(encodeFilename(frag_sanitized))
else:
frags_filenames.append(frag_sanitized)
except (compat_urllib_error.HTTPError, ) as err:
if live and (err.code == 404 or err.code == 410):
# We didn't keep up with the live window. Continue
@ -436,7 +435,4 @@ class F4mFD(FragmentFD):
self._finish_frag_download(ctx)
for frag_file in frags_filenames:
os.remove(encodeFilename(frag_file))
return True

View File

@ -2,6 +2,7 @@ from __future__ import division, unicode_literals
import os
import time
import json
from .common import FileDownloader
from .http import HttpFD
@ -28,15 +29,37 @@ class FragmentFD(FileDownloader):
and hlsnative only)
skip_unavailable_fragments:
Skip unavailable fragments (DASH and hlsnative only)
keep_fragments: Keep downloaded fragments on disk after downloading is
finished
For each incomplete fragment download youtube-dl keeps on disk a special
bookkeeping file with download state and metadata (in future such files will
be used for any incomplete download handled by youtube-dl). This file is
used to properly handle resuming, check download file consistency and detect
potential errors. The file has a .ytdl extension and represents a standard
JSON file of the following format:
extractor:
Dictionary of extractor related data. TBD.
downloader:
Dictionary of downloader related data. May contain following data:
current_fragment:
Dictionary with current (being downloaded) fragment data:
index: 0-based index of current fragment among all fragments
fragment_count:
Total count of fragments
This feature is experimental and file format may change in future.
"""
def report_retry_fragment(self, err, fragment_name, count, retries):
def report_retry_fragment(self, err, frag_index, count, retries):
self.to_screen(
'[download] Got server HTTP error: %s. Retrying fragment %s (attempt %d of %s)...'
% (error_to_compat_str(err), fragment_name, count, self.format_retries(retries)))
'[download] Got server HTTP error: %s. Retrying fragment %d (attempt %d of %s)...'
% (error_to_compat_str(err), frag_index, count, self.format_retries(retries)))
def report_skip_fragment(self, fragment_name):
self.to_screen('[download] Skipping fragment %s...' % fragment_name)
def report_skip_fragment(self, frag_index):
self.to_screen('[download] Skipping fragment %d...' % frag_index)
def _prepare_url(self, info_dict, url):
headers = info_dict.get('http_headers')
@ -46,12 +69,72 @@ class FragmentFD(FileDownloader):
self._prepare_frag_download(ctx)
self._start_frag_download(ctx)
@staticmethod
def __do_ytdl_file(ctx):
return not ctx['live'] and not ctx['tmpfilename'] == '-'
def _read_ytdl_file(self, ctx):
assert 'ytdl_corrupt' not in ctx
stream, _ = sanitize_open(self.ytdl_filename(ctx['filename']), 'r')
try:
ctx['fragment_index'] = json.loads(stream.read())['downloader']['current_fragment']['index']
except Exception:
ctx['ytdl_corrupt'] = True
finally:
stream.close()
def _write_ytdl_file(self, ctx):
frag_index_stream, _ = sanitize_open(self.ytdl_filename(ctx['filename']), 'w')
downloader = {
'current_fragment': {
'index': ctx['fragment_index'],
},
}
if ctx.get('fragment_count') is not None:
downloader['fragment_count'] = ctx['fragment_count']
frag_index_stream.write(json.dumps({'downloader': downloader}))
frag_index_stream.close()
def _download_fragment(self, ctx, frag_url, info_dict, headers=None):
fragment_filename = '%s-Frag%d' % (ctx['tmpfilename'], ctx['fragment_index'])
fragment_info_dict = {
'url': frag_url,
'http_headers': headers or info_dict.get('http_headers'),
}
success = ctx['dl'].download(fragment_filename, fragment_info_dict)
if not success:
return False, None
if fragment_info_dict.get('filetime'):
ctx['fragment_filetime'] = fragment_info_dict.get('filetime')
down, frag_sanitized = sanitize_open(fragment_filename, 'rb')
ctx['fragment_filename_sanitized'] = frag_sanitized
frag_content = down.read()
down.close()
return True, frag_content
def _append_fragment(self, ctx, frag_content):
try:
ctx['dest_stream'].write(frag_content)
ctx['dest_stream'].flush()
finally:
if self.__do_ytdl_file(ctx):
self._write_ytdl_file(ctx)
if not self.params.get('keep_fragments', False):
os.remove(encodeFilename(ctx['fragment_filename_sanitized']))
del ctx['fragment_filename_sanitized']
def _prepare_frag_download(self, ctx):
if 'live' not in ctx:
ctx['live'] = False
if not ctx['live']:
total_frags_str = '%d' % ctx['total_frags']
ad_frags = ctx.get('ad_frags', 0)
if ad_frags:
total_frags_str += ' (not including %d ad)' % ad_frags
else:
total_frags_str = 'unknown (live)'
self.to_screen(
'[%s] Total fragments: %s'
% (self.FD_NAME, ctx['total_frags'] if not ctx['live'] else 'unknown (live)'))
'[%s] Total fragments: %s' % (self.FD_NAME, total_frags_str))
self.report_destination(ctx['filename'])
dl = HttpQuietDownloader(
self.ydl,
@ -66,22 +149,59 @@ class FragmentFD(FileDownloader):
}
)
tmpfilename = self.temp_name(ctx['filename'])
dest_stream, tmpfilename = sanitize_open(tmpfilename, 'wb')
open_mode = 'wb'
resume_len = 0
# Establish possible resume length
if os.path.isfile(encodeFilename(tmpfilename)):
open_mode = 'ab'
resume_len = os.path.getsize(encodeFilename(tmpfilename))
# Should be initialized before ytdl file check
ctx.update({
'tmpfilename': tmpfilename,
'fragment_index': 0,
})
if self.__do_ytdl_file(ctx):
if os.path.isfile(encodeFilename(self.ytdl_filename(ctx['filename']))):
self._read_ytdl_file(ctx)
is_corrupt = ctx.get('ytdl_corrupt') is True
is_inconsistent = ctx['fragment_index'] > 0 and resume_len == 0
if is_corrupt or is_inconsistent:
message = (
'.ytdl file is corrupt' if is_corrupt else
'Inconsistent state of incomplete fragment download')
self.report_warning(
'%s. Restarting from the beginning...' % message)
ctx['fragment_index'] = resume_len = 0
if 'ytdl_corrupt' in ctx:
del ctx['ytdl_corrupt']
self._write_ytdl_file(ctx)
else:
self._write_ytdl_file(ctx)
assert ctx['fragment_index'] == 0
dest_stream, tmpfilename = sanitize_open(tmpfilename, open_mode)
ctx.update({
'dl': dl,
'dest_stream': dest_stream,
'tmpfilename': tmpfilename,
# Total complete fragments downloaded so far in bytes
'complete_frags_downloaded_bytes': resume_len,
})
def _start_frag_download(self, ctx):
resume_len = ctx['complete_frags_downloaded_bytes']
total_frags = ctx['total_frags']
# This dict stores the download progress, it's updated by the progress
# hook
state = {
'status': 'downloading',
'downloaded_bytes': 0,
'frag_index': 0,
'frag_count': total_frags,
'downloaded_bytes': resume_len,
'fragment_index': ctx['fragment_index'],
'fragment_count': total_frags,
'filename': ctx['filename'],
'tmpfilename': ctx['tmpfilename'],
}
@ -89,8 +209,6 @@ class FragmentFD(FileDownloader):
start = time.time()
ctx.update({
'started': start,
# Total complete fragments downloaded so far in bytes
'complete_frags_downloaded_bytes': 0,
# Amount of fragment's bytes downloaded by the time of the previous
# frag progress hook invocation
'prev_frag_downloaded_bytes': 0,
@ -105,12 +223,13 @@ class FragmentFD(FileDownloader):
frag_total_bytes = s.get('total_bytes') or 0
if not ctx['live']:
estimated_size = (
(ctx['complete_frags_downloaded_bytes'] + frag_total_bytes) /
(state['frag_index'] + 1) * total_frags)
(ctx['complete_frags_downloaded_bytes'] + frag_total_bytes)
/ (state['fragment_index'] + 1) * total_frags)
state['total_bytes_estimate'] = estimated_size
if s['status'] == 'finished':
state['frag_index'] += 1
state['fragment_index'] += 1
ctx['fragment_index'] = state['fragment_index']
state['downloaded_bytes'] += frag_total_bytes - ctx['prev_frag_downloaded_bytes']
ctx['complete_frags_downloaded_bytes'] = state['downloaded_bytes']
ctx['prev_frag_downloaded_bytes'] = 0
@ -119,8 +238,8 @@ class FragmentFD(FileDownloader):
state['downloaded_bytes'] += frag_downloaded_bytes - ctx['prev_frag_downloaded_bytes']
if not ctx['live']:
state['eta'] = self.calc_eta(
start, time_now, estimated_size,
state['downloaded_bytes'])
start, time_now, estimated_size - resume_len,
state['downloaded_bytes'] - resume_len)
state['speed'] = s.get('speed') or ctx.get('speed')
ctx['speed'] = state['speed']
ctx['prev_frag_downloaded_bytes'] = frag_downloaded_bytes
@ -132,13 +251,28 @@ class FragmentFD(FileDownloader):
def _finish_frag_download(self, ctx):
ctx['dest_stream'].close()
if self.__do_ytdl_file(ctx):
ytdl_filename = encodeFilename(self.ytdl_filename(ctx['filename']))
if os.path.isfile(ytdl_filename):
os.remove(ytdl_filename)
elapsed = time.time() - ctx['started']
if ctx['tmpfilename'] == '-':
downloaded_bytes = ctx['complete_frags_downloaded_bytes']
else:
self.try_rename(ctx['tmpfilename'], ctx['filename'])
fsize = os.path.getsize(encodeFilename(ctx['filename']))
if self.params.get('updatetime', True):
filetime = ctx.get('fragment_filetime')
if filetime:
try:
os.utime(ctx['filename'], (time.time(), filetime))
except Exception:
pass
downloaded_bytes = os.path.getsize(encodeFilename(ctx['filename']))
self._hook_progress({
'downloaded_bytes': fsize,
'total_bytes': fsize,
'downloaded_bytes': downloaded_bytes,
'total_bytes': downloaded_bytes,
'filename': ctx['filename'],
'status': 'finished',
'elapsed': elapsed,

View File

@ -1,6 +1,5 @@
from __future__ import unicode_literals
import os.path
import re
import binascii
try:
@ -18,8 +17,6 @@ from ..compat import (
compat_struct_pack,
)
from ..utils import (
encodeFilename,
sanitize_open,
parse_m3u8_attributes,
update_url_query,
)
@ -34,7 +31,7 @@ class HlsFD(FragmentFD):
def can_download(manifest, info_dict):
UNSUPPORTED_FEATURES = (
r'#EXT-X-KEY:METHOD=(?!NONE|AES-128)', # encrypted streams [1]
r'#EXT-X-BYTERANGE', # playlists composed of byte ranges of media files [2]
# r'#EXT-X-BYTERANGE', # playlists composed of byte ranges of media files [2]
# Live streams heuristic does not always work (e.g. geo restricted to Germany
# http://hls-geo.daserste.de/i/videoportal/Film/c_620000/622873/format,716451,716457,716450,716458,716459,.mp4.csmil/index_4_av.m3u8?null=0)
@ -52,7 +49,9 @@ class HlsFD(FragmentFD):
# 4. https://tools.ietf.org/html/draft-pantos-http-live-streaming-17#section-4.3.3.5
)
check_results = [not re.search(feature, manifest) for feature in UNSUPPORTED_FEATURES]
check_results.append(can_decrypt_frag or '#EXT-X-KEY:METHOD=AES-128' not in manifest)
is_aes128_enc = '#EXT-X-KEY:METHOD=AES-128' in manifest
check_results.append(can_decrypt_frag or not is_aes128_enc)
check_results.append(not (is_aes128_enc and r'#EXT-X-BYTERANGE' in manifest))
check_results.append(not info_dict.get('is_live'))
return all(check_results)
@ -60,12 +59,12 @@ class HlsFD(FragmentFD):
man_url = info_dict['url']
self.to_screen('[%s] Downloading m3u8 manifest' % self.FD_NAME)
manifest = self.ydl.urlopen(self._prepare_url(info_dict, man_url)).read()
s = manifest.decode('utf-8', 'ignore')
urlh = self.ydl.urlopen(self._prepare_url(info_dict, man_url))
man_url = urlh.geturl()
s = urlh.read().decode('utf-8', 'ignore')
if not self.can_download(s, info_dict):
if info_dict.get('extra_param_to_segment_url'):
if info_dict.get('extra_param_to_segment_url') or info_dict.get('_decryption_key_url'):
self.report_error('pycrypto not found. Please install it.')
return False
self.report_warning(
@ -76,15 +75,36 @@ class HlsFD(FragmentFD):
fd.add_progress_hook(ph)
return fd.real_download(filename, info_dict)
total_frags = 0
def is_ad_fragment_start(s):
return (s.startswith('#ANVATO-SEGMENT-INFO') and 'type=ad' in s
or s.startswith('#UPLYNK-SEGMENT') and s.endswith(',ad'))
def is_ad_fragment_end(s):
return (s.startswith('#ANVATO-SEGMENT-INFO') and 'type=master' in s
or s.startswith('#UPLYNK-SEGMENT') and s.endswith(',segment'))
media_frags = 0
ad_frags = 0
ad_frag_next = False
for line in s.splitlines():
line = line.strip()
if line and not line.startswith('#'):
total_frags += 1
if not line:
continue
if line.startswith('#'):
if is_ad_fragment_start(line):
ad_frag_next = True
elif is_ad_fragment_end(line):
ad_frag_next = False
continue
if ad_frag_next:
ad_frags += 1
continue
media_frags += 1
ctx = {
'filename': filename,
'total_frags': total_frags,
'total_frags': media_frags,
'ad_frags': ad_frags,
}
self._prepare_and_start_frag_download(ctx)
@ -100,61 +120,66 @@ class HlsFD(FragmentFD):
i = 0
media_sequence = 0
decrypt_info = {'METHOD': 'NONE'}
frags_filenames = []
byte_range = {}
frag_index = 0
ad_frag_next = False
for line in s.splitlines():
line = line.strip()
if line:
if not line.startswith('#'):
if ad_frag_next:
continue
frag_index += 1
if frag_index <= ctx['fragment_index']:
continue
frag_url = (
line
if re.match(r'^https?://', line)
else compat_urlparse.urljoin(man_url, line))
frag_name = 'Frag%d' % i
frag_filename = '%s-%s' % (ctx['tmpfilename'], frag_name)
if extra_query:
frag_url = update_url_query(frag_url, extra_query)
count = 0
headers = info_dict.get('http_headers', {})
if byte_range:
headers['Range'] = 'bytes=%d-%d' % (byte_range['start'], byte_range['end'] - 1)
while count <= fragment_retries:
try:
success = ctx['dl'].download(frag_filename, {
'url': frag_url,
'http_headers': info_dict.get('http_headers'),
})
success, frag_content = self._download_fragment(
ctx, frag_url, info_dict, headers)
if not success:
return False
down, frag_sanitized = sanitize_open(frag_filename, 'rb')
frag_content = down.read()
down.close()
break
except compat_urllib_error.HTTPError as err:
# Unavailable (possibly temporary) fragments may be served.
# First we try to retry then either skip or abort.
# See https://github.com/rg3/youtube-dl/issues/10165,
# https://github.com/rg3/youtube-dl/issues/10448).
# See https://github.com/ytdl-org/youtube-dl/issues/10165,
# https://github.com/ytdl-org/youtube-dl/issues/10448).
count += 1
if count <= fragment_retries:
self.report_retry_fragment(err, frag_name, count, fragment_retries)
self.report_retry_fragment(err, frag_index, count, fragment_retries)
if count > fragment_retries:
if skip_unavailable_fragments:
i += 1
media_sequence += 1
self.report_skip_fragment(frag_name)
self.report_skip_fragment(frag_index)
continue
self.report_error(
'giving up after %s fragment retries' % fragment_retries)
return False
if decrypt_info['METHOD'] == 'AES-128':
iv = decrypt_info.get('IV') or compat_struct_pack('>8xq', media_sequence)
decrypt_info['KEY'] = decrypt_info.get('KEY') or self.ydl.urlopen(
self._prepare_url(info_dict, info_dict.get('_decryption_key_url') or decrypt_info['URI'])).read()
frag_content = AES.new(
decrypt_info['KEY'], AES.MODE_CBC, iv).decrypt(frag_content)
ctx['dest_stream'].write(frag_content)
frags_filenames.append(frag_sanitized)
self._append_fragment(ctx, frag_content)
# We only download the first fragment during the test
if test:
break
i += 1
media_sequence += 1
elif line.startswith('#EXT-X-KEY'):
decrypt_url = decrypt_info.get('URI')
decrypt_info = parse_m3u8_attributes(line[11:])
if decrypt_info['METHOD'] == 'AES-128':
if 'IV' in decrypt_info:
@ -164,13 +189,22 @@ class HlsFD(FragmentFD):
man_url, decrypt_info['URI'])
if extra_query:
decrypt_info['URI'] = update_url_query(decrypt_info['URI'], extra_query)
decrypt_info['KEY'] = self.ydl.urlopen(decrypt_info['URI']).read()
if decrypt_url != decrypt_info['URI']:
decrypt_info['KEY'] = None
elif line.startswith('#EXT-X-MEDIA-SEQUENCE'):
media_sequence = int(line[22:])
elif line.startswith('#EXT-X-BYTERANGE'):
splitted_byte_range = line[17:].split('@')
sub_range_start = int(splitted_byte_range[1]) if len(splitted_byte_range) == 2 else byte_range['end']
byte_range = {
'start': sub_range_start,
'end': sub_range_start + int(splitted_byte_range[0]),
}
elif is_ad_fragment_start(line):
ad_frag_next = True
elif is_ad_fragment_end(line):
ad_frag_next = False
self._finish_frag_download(ctx)
for frag_file in frags_filenames:
os.remove(encodeFilename(frag_file))
return True

View File

@ -4,13 +4,18 @@ import errno
import os
import socket
import time
import random
import re
from .common import FileDownloader
from ..compat import compat_urllib_error
from ..compat import (
compat_str,
compat_urllib_error,
)
from ..utils import (
ContentTooShortError,
encodeFilename,
int_or_none,
sanitize_open,
sanitized_Request,
write_xattr,
@ -22,79 +27,140 @@ from ..utils import (
class HttpFD(FileDownloader):
def real_download(self, filename, info_dict):
url = info_dict['url']
tmpfilename = self.temp_name(filename)
stream = None
class DownloadContext(dict):
__getattr__ = dict.get
__setattr__ = dict.__setitem__
__delattr__ = dict.__delitem__
ctx = DownloadContext()
ctx.filename = filename
ctx.tmpfilename = self.temp_name(filename)
ctx.stream = None
# Do not include the Accept-Encoding header
headers = {'Youtubedl-no-compression': 'True'}
add_headers = info_dict.get('http_headers')
if add_headers:
headers.update(add_headers)
basic_request = sanitized_Request(url, None, headers)
request = sanitized_Request(url, None, headers)
is_test = self.params.get('test', False)
chunk_size = self._TEST_FILE_SIZE if is_test else (
info_dict.get('downloader_options', {}).get('http_chunk_size')
or self.params.get('http_chunk_size') or 0)
if is_test:
request.add_header('Range', 'bytes=0-%s' % str(self._TEST_FILE_SIZE - 1))
ctx.open_mode = 'wb'
ctx.resume_len = 0
ctx.data_len = None
ctx.block_size = self.params.get('buffersize', 1024)
ctx.start_time = time.time()
ctx.chunk_size = None
# Establish possible resume length
if os.path.isfile(encodeFilename(tmpfilename)):
resume_len = os.path.getsize(encodeFilename(tmpfilename))
else:
resume_len = 0
open_mode = 'wb'
if resume_len != 0:
if self.params.get('continuedl', True):
self.report_resuming_byte(resume_len)
request.add_header('Range', 'bytes=%d-' % resume_len)
open_mode = 'ab'
else:
resume_len = 0
# Establish possible resume length
if os.path.isfile(encodeFilename(ctx.tmpfilename)):
ctx.resume_len = os.path.getsize(
encodeFilename(ctx.tmpfilename))
ctx.is_resume = ctx.resume_len > 0
count = 0
retries = self.params.get('retries', 0)
while count <= retries:
class SucceedDownload(Exception):
pass
class RetryDownload(Exception):
def __init__(self, source_error):
self.source_error = source_error
class NextFragment(Exception):
pass
def set_range(req, start, end):
range_header = 'bytes=%d-' % start
if end:
range_header += compat_str(end)
req.add_header('Range', range_header)
def establish_connection():
ctx.chunk_size = (random.randint(int(chunk_size * 0.95), chunk_size)
if not is_test and chunk_size else chunk_size)
if ctx.resume_len > 0:
range_start = ctx.resume_len
if ctx.is_resume:
self.report_resuming_byte(ctx.resume_len)
ctx.open_mode = 'ab'
elif ctx.chunk_size > 0:
range_start = 0
else:
range_start = None
ctx.is_resume = False
range_end = range_start + ctx.chunk_size - 1 if ctx.chunk_size else None
if range_end and ctx.data_len is not None and range_end >= ctx.data_len:
range_end = ctx.data_len - 1
has_range = range_start is not None
ctx.has_range = has_range
request = sanitized_Request(url, None, headers)
if has_range:
set_range(request, range_start, range_end)
# Establish connection
try:
data = self.ydl.urlopen(request)
try:
ctx.data = self.ydl.urlopen(request)
except (compat_urllib_error.URLError, ) as err:
# reason may not be available, e.g. for urllib2.HTTPError on python 2.6
reason = getattr(err, 'reason', None)
if isinstance(reason, socket.timeout):
raise RetryDownload(err)
raise err
# When trying to resume, Content-Range HTTP header of response has to be checked
# to match the value of requested Range HTTP header. This is due to a webservers
# that don't support resuming and serve a whole file with no Content-Range
# set in response despite of requested Range (see
# https://github.com/rg3/youtube-dl/issues/6057#issuecomment-126129799)
if resume_len > 0:
content_range = data.headers.get('Content-Range')
# https://github.com/ytdl-org/youtube-dl/issues/6057#issuecomment-126129799)
if has_range:
content_range = ctx.data.headers.get('Content-Range')
if content_range:
content_range_m = re.search(r'bytes (\d+)-', content_range)
content_range_m = re.search(r'bytes (\d+)-(\d+)?(?:/(\d+))?', content_range)
# Content-Range is present and matches requested Range, resume is possible
if content_range_m and resume_len == int(content_range_m.group(1)):
break
if content_range_m:
if range_start == int(content_range_m.group(1)):
content_range_end = int_or_none(content_range_m.group(2))
content_len = int_or_none(content_range_m.group(3))
accept_content_len = (
# Non-chunked download
not ctx.chunk_size
# Chunked download and requested piece or
# its part is promised to be served
or content_range_end == range_end
or content_len < range_end)
if accept_content_len:
ctx.data_len = content_len
return
# Content-Range is either not present or invalid. Assuming remote webserver is
# trying to send the whole file, resume is not possible, so wiping the local file
# and performing entire redownload
self.report_unable_to_resume()
resume_len = 0
open_mode = 'wb'
break
ctx.resume_len = 0
ctx.open_mode = 'wb'
ctx.data_len = int_or_none(ctx.data.info().get('Content-length', None))
return
except (compat_urllib_error.HTTPError, ) as err:
if (err.code < 500 or err.code >= 600) and err.code != 416:
# Unexpected HTTP error
raise
elif err.code == 416:
if err.code == 416:
# Unable to resume (requested range not satisfiable)
try:
# Open the connection again without the range header
data = self.ydl.urlopen(basic_request)
content_length = data.info()['Content-Length']
ctx.data = self.ydl.urlopen(
sanitized_Request(url, None, headers))
content_length = ctx.data.info()['Content-Length']
except (compat_urllib_error.HTTPError, ) as err:
if err.code < 500 or err.code >= 600:
raise
else:
# Examine the reported length
if (content_length is not None and
(resume_len - 100 < int(content_length) < resume_len + 100)):
if (content_length is not None
and (ctx.resume_len - 100 < int(content_length) < ctx.resume_len + 100)):
# The file had already been fully downloaded.
# Explanation to the above condition: in issue #175 it was revealed that
# YouTube sometimes adds or removes a few bytes from the end of the file,
@ -102,36 +168,33 @@ class HttpFD(FileDownloader):
# I decided to implement a suggested change and consider the file
# completely downloaded if the file size differs less than 100 bytes from
# the one in the hard drive.
self.report_file_already_downloaded(filename)
self.try_rename(tmpfilename, filename)
self.report_file_already_downloaded(ctx.filename)
self.try_rename(ctx.tmpfilename, ctx.filename)
self._hook_progress({
'filename': filename,
'filename': ctx.filename,
'status': 'finished',
'downloaded_bytes': resume_len,
'total_bytes': resume_len,
'downloaded_bytes': ctx.resume_len,
'total_bytes': ctx.resume_len,
})
return True
raise SucceedDownload()
else:
# The length does not match, we start the download over
self.report_unable_to_resume()
resume_len = 0
open_mode = 'wb'
break
except socket.error as e:
if e.errno != errno.ECONNRESET:
ctx.resume_len = 0
ctx.open_mode = 'wb'
return
elif err.code < 500 or err.code >= 600:
# Unexpected HTTP error
raise
raise RetryDownload(err)
except socket.error as err:
if err.errno != errno.ECONNRESET:
# Connection reset is no problem, just retry
raise
raise RetryDownload(err)
# Retry
count += 1
if count <= retries:
self.report_retry(count, retries)
if count > retries:
self.report_error('giving up after %s retries' % retries)
return False
data_len = data.info().get('Content-length', None)
def download():
data_len = ctx.data.info().get('Content-length', None)
# Range HTTP header may be ignored/unsupported by a webserver
# (e.g. extractor/scivee.py, extractor/bambuser.py).
@ -142,7 +205,7 @@ class HttpFD(FileDownloader):
data_len = self._TEST_FILE_SIZE
if data_len is not None:
data_len = int(data_len) + resume_len
data_len = int(data_len) + ctx.resume_len
min_data_len = self.params.get('min_filesize')
max_data_len = self.params.get('max_filesize')
if min_data_len is not None and data_len < min_data_len:
@ -152,17 +215,38 @@ class HttpFD(FileDownloader):
self.to_screen('\r[download] File is larger than max-filesize (%s bytes > %s bytes). Aborting.' % (data_len, max_data_len))
return False
byte_counter = 0 + resume_len
block_size = self.params.get('buffersize', 1024)
byte_counter = 0 + ctx.resume_len
block_size = ctx.block_size
start = time.time()
# measure time over whole while-loop, so slow_down() and best_block_size() work together properly
now = None # needed for slow_down() in the first loop run
before = start # start measuring
while True:
def retry(e):
to_stdout = ctx.tmpfilename == '-'
if ctx.stream is not None:
if not to_stdout:
ctx.stream.close()
ctx.stream = None
ctx.resume_len = byte_counter if to_stdout else os.path.getsize(encodeFilename(ctx.tmpfilename))
raise RetryDownload(e)
while True:
try:
# Download and write
data_block = data.read(block_size if not is_test else min(block_size, data_len - byte_counter))
data_block = ctx.data.read(block_size if data_len is None else min(block_size, data_len - byte_counter))
# socket.timeout is a subclass of socket.error but may not have
# errno set
except socket.timeout as e:
retry(e)
except socket.error as e:
# SSLError on python 2 (inherits socket.error) may have
# no errno set but this error message
if e.errno in (errno.ECONNRESET, errno.ETIMEDOUT) or getattr(e, 'message', None) == 'The read operation timed out':
retry(e)
raise
byte_counter += len(data_block)
# exit loop when download is finished
@ -170,31 +254,32 @@ class HttpFD(FileDownloader):
break
# Open destination file just in time
if stream is None:
if ctx.stream is None:
try:
(stream, tmpfilename) = sanitize_open(tmpfilename, open_mode)
assert stream is not None
filename = self.undo_temp_name(tmpfilename)
self.report_destination(filename)
ctx.stream, ctx.tmpfilename = sanitize_open(
ctx.tmpfilename, ctx.open_mode)
assert ctx.stream is not None
ctx.filename = self.undo_temp_name(ctx.tmpfilename)
self.report_destination(ctx.filename)
except (OSError, IOError) as err:
self.report_error('unable to open for writing: %s' % str(err))
return False
if self.params.get('xattr_set_filesize', False) and data_len is not None:
try:
write_xattr(tmpfilename, 'user.ytdl.filesize', str(data_len).encode('utf-8'))
write_xattr(ctx.tmpfilename, 'user.ytdl.filesize', str(data_len).encode('utf-8'))
except (XAttrUnavailableError, XAttrMetadataError) as err:
self.report_error('unable to set filesize xattr: %s' % str(err))
try:
stream.write(data_block)
ctx.stream.write(data_block)
except (IOError, OSError) as err:
self.to_stderr('\n')
self.report_error('unable to write data: %s' % str(err))
return False
# Apply rate limit
self.slow_down(start, now, byte_counter - resume_len)
self.slow_down(start, now, byte_counter - ctx.resume_len)
# end measuring of one loop run
now = time.time()
@ -207,47 +292,73 @@ class HttpFD(FileDownloader):
before = after
# Progress message
speed = self.calc_speed(start, now, byte_counter - resume_len)
if data_len is None:
speed = self.calc_speed(start, now, byte_counter - ctx.resume_len)
if ctx.data_len is None:
eta = None
else:
eta = self.calc_eta(start, time.time(), data_len - resume_len, byte_counter - resume_len)
eta = self.calc_eta(start, time.time(), ctx.data_len - ctx.resume_len, byte_counter - ctx.resume_len)
self._hook_progress({
'status': 'downloading',
'downloaded_bytes': byte_counter,
'total_bytes': data_len,
'tmpfilename': tmpfilename,
'filename': filename,
'total_bytes': ctx.data_len,
'tmpfilename': ctx.tmpfilename,
'filename': ctx.filename,
'eta': eta,
'speed': speed,
'elapsed': now - start,
'elapsed': now - ctx.start_time,
})
if is_test and byte_counter == data_len:
if data_len is not None and byte_counter == data_len:
break
if stream is None:
if not is_test and ctx.chunk_size and ctx.data_len is not None and byte_counter < ctx.data_len:
ctx.resume_len = byte_counter
# ctx.block_size = block_size
raise NextFragment()
if ctx.stream is None:
self.to_stderr('\n')
self.report_error('Did not get any data blocks')
return False
if tmpfilename != '-':
stream.close()
if ctx.tmpfilename != '-':
ctx.stream.close()
if data_len is not None and byte_counter != data_len:
raise ContentTooShortError(byte_counter, int(data_len))
self.try_rename(tmpfilename, filename)
err = ContentTooShortError(byte_counter, int(data_len))
if count <= retries:
retry(err)
raise err
self.try_rename(ctx.tmpfilename, ctx.filename)
# Update file modification time
if self.params.get('updatetime', True):
info_dict['filetime'] = self.try_utime(filename, data.info().get('last-modified', None))
info_dict['filetime'] = self.try_utime(ctx.filename, ctx.data.info().get('last-modified', None))
self._hook_progress({
'downloaded_bytes': byte_counter,
'total_bytes': byte_counter,
'filename': filename,
'filename': ctx.filename,
'status': 'finished',
'elapsed': time.time() - start,
'elapsed': time.time() - ctx.start_time,
})
return True
while count <= retries:
try:
establish_connection()
return download()
except RetryDownload as e:
count += 1
if count <= retries:
self.report_retry(e.source_error, count, retries)
continue
except NextFragment:
continue
except SucceedDownload:
return True
self.report_error('giving up after %s retries' % retries)
return False

View File

@ -1,30 +1,27 @@
from __future__ import unicode_literals
import os
import time
import struct
import binascii
import io
from .fragment import FragmentFD
from ..compat import compat_urllib_error
from ..utils import (
sanitize_open,
encodeFilename,
from ..compat import (
compat_Struct,
compat_urllib_error,
)
u8 = struct.Struct(b'>B')
u88 = struct.Struct(b'>Bx')
u16 = struct.Struct(b'>H')
u1616 = struct.Struct(b'>Hxx')
u32 = struct.Struct(b'>I')
u64 = struct.Struct(b'>Q')
u8 = compat_Struct('>B')
u88 = compat_Struct('>Bx')
u16 = compat_Struct('>H')
u1616 = compat_Struct('>Hxx')
u32 = compat_Struct('>I')
u64 = compat_Struct('>Q')
s88 = struct.Struct(b'>bx')
s16 = struct.Struct(b'>h')
s1616 = struct.Struct(b'>hxx')
s32 = struct.Struct(b'>i')
s88 = compat_Struct('>bx')
s16 = compat_Struct('>h')
s1616 = compat_Struct('>hxx')
s32 = compat_Struct('>i')
unity_matrix = (s32.pack(0x10000) + s32.pack(0) * 3) * 2 + s32.pack(0x40000000)
@ -103,7 +100,7 @@ def write_piff_header(stream, params):
if is_audio:
smhd_payload = s88.pack(0) # balance
smhd_payload = u16.pack(0) # reserved
smhd_payload += u16.pack(0) # reserved
media_header_box = full_box(b'smhd', 0, 0, smhd_payload) # Sound Media Header
else:
vmhd_payload = u16.pack(0) # graphics mode
@ -131,7 +128,6 @@ def write_piff_header(stream, params):
if fourcc == 'AACL':
sample_entry_box = box(b'mp4a', sample_entry_payload)
else:
sample_entry_payload = sample_entry_payload
sample_entry_payload += u16.pack(0) # pre defined
sample_entry_payload += u16.pack(0) # reserved
sample_entry_payload += u32.pack(0) * 3 # pre defined
@ -145,12 +141,12 @@ def write_piff_header(stream, params):
sample_entry_payload += u16.pack(0x18) # depth
sample_entry_payload += s16.pack(-1) # pre defined
codec_private_data = binascii.unhexlify(params['codec_private_data'])
codec_private_data = binascii.unhexlify(params['codec_private_data'].encode('utf-8'))
if fourcc in ('H264', 'AVC1'):
sps, pps = codec_private_data.split(u32.pack(1))[1:]
avcc_payload = u8.pack(1) # configuration version
avcc_payload += sps[1:4] # avc profile indication + profile compatibility + avc level indication
avcc_payload += u8.pack(0xfc | (params.get('nal_unit_length_field', 4) - 1)) # complete represenation (1) + reserved (11111) + length size minus one
avcc_payload += u8.pack(0xfc | (params.get('nal_unit_length_field', 4) - 1)) # complete representation (1) + reserved (11111) + length size minus one
avcc_payload += u8.pack(1) # reserved (0) + number of sps (0000001)
avcc_payload += u16.pack(len(sps))
avcc_payload += sps
@ -225,47 +221,39 @@ class IsmFD(FragmentFD):
self._prepare_and_start_frag_download(ctx)
segments_filenames = []
fragment_retries = self.params.get('fragment_retries', 0)
skip_unavailable_fragments = self.params.get('skip_unavailable_fragments', True)
track_written = False
frag_index = 0
for i, segment in enumerate(segments):
segment_url = segment['url']
segment_name = 'Frag%d' % i
target_filename = '%s-%s' % (ctx['tmpfilename'], segment_name)
frag_index += 1
if frag_index <= ctx['fragment_index']:
continue
count = 0
while count <= fragment_retries:
try:
success = ctx['dl'].download(target_filename, {'url': segment_url})
success, frag_content = self._download_fragment(ctx, segment['url'], info_dict)
if not success:
return False
down, target_sanitized = sanitize_open(target_filename, 'rb')
down_data = down.read()
if not track_written:
tfhd_data = extract_box_data(down_data, [b'moof', b'traf', b'tfhd'])
tfhd_data = extract_box_data(frag_content, [b'moof', b'traf', b'tfhd'])
info_dict['_download_params']['track_id'] = u32.unpack(tfhd_data[4:8])[0]
write_piff_header(ctx['dest_stream'], info_dict['_download_params'])
track_written = True
ctx['dest_stream'].write(down_data)
down.close()
segments_filenames.append(target_sanitized)
self._append_fragment(ctx, frag_content)
break
except compat_urllib_error.HTTPError as err:
count += 1
if count <= fragment_retries:
self.report_retry_fragment(err, segment_name, count, fragment_retries)
self.report_retry_fragment(err, frag_index, count, fragment_retries)
if count > fragment_retries:
if skip_unavailable_fragments:
self.report_skip_fragment(segment_name)
self.report_skip_fragment(frag_index)
continue
self.report_error('giving up after %s fragment retries' % fragment_retries)
return False
self._finish_frag_download(ctx)
for segment_file in segments_filenames:
os.remove(encodeFilename(segment_file))
return True

View File

@ -29,6 +29,7 @@ class RtmpFD(FileDownloader):
proc = subprocess.Popen(args, stderr=subprocess.PIPE)
cursor_in_new_line = True
proc_stderr_closed = False
try:
while not proc_stderr_closed:
# read line from stderr
line = ''
@ -88,6 +89,7 @@ class RtmpFD(FileDownloader):
self.to_screen('')
cursor_in_new_line = True
self.to_screen('[rtmpdump] ' + line)
finally:
proc.wait()
if not cursor_in_new_line:
self.to_screen('')
@ -163,15 +165,23 @@ class RtmpFD(FileDownloader):
RD_INCOMPLETE = 2
RD_NO_CONNECT = 3
started = time.time()
try:
retval = run_rtmpdump(args)
except KeyboardInterrupt:
if not info_dict.get('is_live'):
raise
retval = RD_SUCCESS
self.to_screen('\n[rtmpdump] Interrupted by user')
if retval == RD_NO_CONNECT:
self.report_error('[rtmpdump] Could not connect to RTMP server.')
return False
while (retval == RD_INCOMPLETE or retval == RD_FAILED) and not test and not live:
while retval in (RD_INCOMPLETE, RD_FAILED) and not test and not live:
prevsize = os.path.getsize(encodeFilename(tmpfilename))
self.to_screen('[rtmpdump] %s bytes' % prevsize)
self.to_screen('[rtmpdump] Downloaded %s bytes' % prevsize)
time.sleep(5.0) # This seems to be needed
args = basic_args + ['--resume']
if retval == RD_FAILED:
@ -188,13 +198,14 @@ class RtmpFD(FileDownloader):
break
if retval == RD_SUCCESS or (test and retval == RD_INCOMPLETE):
fsize = os.path.getsize(encodeFilename(tmpfilename))
self.to_screen('[rtmpdump] %s bytes' % fsize)
self.to_screen('[rtmpdump] Downloaded %s bytes' % fsize)
self.try_rename(tmpfilename, filename)
self._hook_progress({
'downloaded_bytes': fsize,
'total_bytes': fsize,
'filename': filename,
'status': 'finished',
'elapsed': time.time() - started,
})
return True
else:

View File

@ -1,13 +1,20 @@
from __future__ import unicode_literals
import hashlib
import hmac
import re
import time
from .common import InfoExtractor
from ..compat import compat_str
from ..utils import (
ExtractorError,
js_to_json,
int_or_none,
parse_iso8601,
try_get,
unescapeHTML,
update_url_query,
)
@ -98,33 +105,59 @@ class ABCIE(InfoExtractor):
class ABCIViewIE(InfoExtractor):
IE_NAME = 'abc.net.au:iview'
_VALID_URL = r'https?://iview\.abc\.net\.au/programs/[^/]+/(?P<id>[^/?#]+)'
_VALID_URL = r'https?://iview\.abc\.net\.au/(?:[^/]+/)*video/(?P<id>[^/?#]+)'
_GEO_COUNTRIES = ['AU']
# ABC iview programs are normally available for 14 days only.
_TESTS = [{
'url': 'http://iview.abc.net.au/programs/diaries-of-a-broken-mind/ZX9735A001S00',
'md5': 'cde42d728b3b7c2b32b1b94b4a548afc',
'url': 'https://iview.abc.net.au/show/gruen/series/11/video/LE1927H001S00',
'md5': '67715ce3c78426b11ba167d875ac6abf',
'info_dict': {
'id': 'ZX9735A001S00',
'id': 'LE1927H001S00',
'ext': 'mp4',
'title': 'Diaries Of A Broken Mind',
'description': 'md5:7de3903874b7a1be279fe6b68718fc9e',
'upload_date': '20161010',
'uploader_id': 'abc2',
'timestamp': 1476064920,
'title': "Series 11 Ep 1",
'series': "Gruen",
'description': 'md5:52cc744ad35045baf6aded2ce7287f67',
'upload_date': '20190925',
'uploader_id': 'abc1',
'timestamp': 1569445289,
},
'params': {
'skip_download': True,
},
'skip': 'Video gone',
}]
def _real_extract(self, url):
video_id = self._match_id(url)
webpage = self._download_webpage(url, video_id)
video_params = self._parse_json(self._search_regex(
r'videoParams\s*=\s*({.+?});', webpage, 'video params'), video_id)
title = video_params.get('title') or video_params['seriesTitle']
stream = next(s for s in video_params['playlist'] if s.get('type') == 'program')
video_params = self._download_json(
'https://iview.abc.net.au/api/programs/' + video_id, video_id)
title = unescapeHTML(video_params.get('title') or video_params['seriesTitle'])
stream = next(s for s in video_params['playlist'] if s.get('type') in ('program', 'livestream'))
formats = self._extract_akamai_formats(stream['hds-unmetered'], video_id)
house_number = video_params.get('episodeHouseNumber') or video_id
path = '/auth/hls/sign?ts={0}&hn={1}&d=android-tablet'.format(
int(time.time()), house_number)
sig = hmac.new(
b'android.content.res.Resources',
path.encode('utf-8'), hashlib.sha256).hexdigest()
token = self._download_webpage(
'http://iview.abc.net.au{0}&sig={1}'.format(path, sig), video_id)
def tokenize_url(url, token):
return update_url_query(url, {
'hdnea': token,
})
for sd in ('720', 'sd', 'sd-low'):
sd_url = try_get(
stream, lambda x: x['streams']['hls'][sd], compat_str)
if not sd_url:
continue
formats = self._extract_m3u8_formats(
tokenize_url(sd_url, token), video_id, 'mp4',
entry_protocol='m3u8_native', m3u8_id='hls', fatal=False)
if formats:
break
self._sort_formats(formats)
subtitles = {}
@ -135,18 +168,26 @@ class ABCIViewIE(InfoExtractor):
'ext': 'vtt',
}]
is_live = video_params.get('livestream') == '1'
if is_live:
title = self._live_title(title)
return {
'id': video_id,
'title': title,
'description': self._html_search_meta(['og:description', 'twitter:description'], webpage),
'thumbnail': self._html_search_meta(['og:image', 'twitter:image:src'], webpage),
'description': video_params.get('description'),
'thumbnail': video_params.get('thumbnail'),
'duration': int_or_none(video_params.get('eventDuration')),
'timestamp': parse_iso8601(video_params.get('pubDate'), ' '),
'series': video_params.get('seriesTitle'),
'series': unescapeHTML(video_params.get('seriesTitle')),
'series_id': video_params.get('seriesHouseNumber') or video_id[:7],
'episode_number': int_or_none(self._html_search_meta('episodeNumber', webpage, default=None)),
'episode': self._html_search_meta('episode_title', webpage, default=None),
'season_number': int_or_none(self._search_regex(
r'\bSeries\s+(\d+)\b', title, 'season number', default=None)),
'episode_number': int_or_none(self._search_regex(
r'\bEp\s+(\d+)\b', title, 'episode number', default=None)),
'episode_id': house_number,
'uploader_id': video_params.get('channel'),
'formats': formats,
'subtitles': subtitles,
'is_live': is_live,
}

View File

@ -7,12 +7,24 @@ import time
from .amp import AMPIE
from .common import InfoExtractor
from .youtube import YoutubeIE
from ..compat import compat_urlparse
class AbcNewsVideoIE(AMPIE):
IE_NAME = 'abcnews:video'
_VALID_URL = r'https?://abcnews\.go\.com/[^/]+/video/(?P<display_id>[0-9a-z-]+)-(?P<id>\d+)'
_VALID_URL = r'''(?x)
https?://
(?:
abcnews\.go\.com/
(?:
[^/]+/video/(?P<display_id>[0-9a-z-]+)-|
video/embed\?.*?\bid=
)|
fivethirtyeight\.abcnews\.go\.com/video/embed/\d+/
)
(?P<id>\d+)
'''
_TESTS = [{
'url': 'http://abcnews.go.com/ThisWeek/video/week-exclusive-irans-foreign-minister-zarif-20411932',
@ -29,6 +41,9 @@ class AbcNewsVideoIE(AMPIE):
# m3u8 download
'skip_download': True,
},
}, {
'url': 'http://abcnews.go.com/video/embed?id=46979033',
'only_matching': True,
}, {
'url': 'http://abcnews.go.com/2020/video/2020-husband-stands-teacher-jail-student-affairs-26119478',
'only_matching': True,
@ -54,7 +69,7 @@ class AbcNewsIE(InfoExtractor):
_TESTS = [{
'url': 'http://abcnews.go.com/Blotter/News/dramatic-video-rare-death-job-america/story?id=10498713#.UIhwosWHLjY',
'info_dict': {
'id': '10498713',
'id': '10505354',
'ext': 'flv',
'display_id': 'dramatic-video-rare-death-job-america',
'title': 'Occupational Hazards',
@ -67,7 +82,7 @@ class AbcNewsIE(InfoExtractor):
}, {
'url': 'http://abcnews.go.com/Entertainment/justin-timberlake-performs-stop-feeling-eurovision-2016/story?id=39125818',
'info_dict': {
'id': '39125818',
'id': '38897857',
'ext': 'mp4',
'display_id': 'justin-timberlake-performs-stop-feeling-eurovision-2016',
'title': 'Justin Timberlake Drops Hints For Secret Single',
@ -97,9 +112,7 @@ class AbcNewsIE(InfoExtractor):
r'window\.abcnvideo\.url\s*=\s*"([^"]+)"', webpage, 'video URL')
full_video_url = compat_urlparse.urljoin(url, video_url)
youtube_url = self._html_search_regex(
r'<iframe[^>]+src="(https://www\.youtube\.com/embed/[^"]+)"',
webpage, 'YouTube URL', default=None)
youtube_url = YoutubeIE._extract_url(webpage)
timestamp = None
date_str = self._html_search_regex(
@ -129,7 +142,7 @@ class AbcNewsIE(InfoExtractor):
}
if youtube_url:
entries = [entry, self.url_result(youtube_url, 'Youtube')]
entries = [entry, self.url_result(youtube_url, ie=YoutubeIE.ie_key())]
return self.playlist_result(entries)
return entry

View File

@ -4,29 +4,30 @@ from __future__ import unicode_literals
import re
from .common import InfoExtractor
from ..compat import compat_str
from ..utils import (
dict_get,
int_or_none,
parse_iso8601,
try_get,
)
class ABCOTVSIE(InfoExtractor):
IE_NAME = 'abcotvs'
IE_DESC = 'ABC Owned Television Stations'
_VALID_URL = r'https?://(?:abc(?:7(?:news|ny|chicago)?|11|13|30)|6abc)\.com(?:/[^/]+/(?P<display_id>[^/]+))?/(?P<id>\d+)'
_VALID_URL = r'https?://(?P<site>abc(?:7(?:news|ny|chicago)?|11|13|30)|6abc)\.com(?:(?:/[^/]+)*/(?P<display_id>[^/]+))?/(?P<id>\d+)'
_TESTS = [
{
'url': 'http://abc7news.com/entertainment/east-bay-museum-celebrates-vintage-synthesizers/472581/',
'info_dict': {
'id': '472581',
'id': '472548',
'display_id': 'east-bay-museum-celebrates-vintage-synthesizers',
'ext': 'mp4',
'title': 'East Bay museum celebrates vintage synthesizers',
'description': 'md5:a4f10fb2f2a02565c1749d4adbab4b10',
'title': 'East Bay museum celebrates synthesized music',
'description': 'md5:24ed2bd527096ec2a5c67b9d5a9005f3',
'thumbnail': r're:^https?://.*\.jpg$',
'timestamp': 1421123075,
'timestamp': 1421118520,
'upload_date': '20150113',
'uploader': 'Jonathan Bloom',
},
'params': {
# m3u8 download
@ -37,39 +38,63 @@ class ABCOTVSIE(InfoExtractor):
'url': 'http://abc7news.com/472581',
'only_matching': True,
},
{
'url': 'https://6abc.com/man-75-killed-after-being-struck-by-vehicle-in-chester/5725182/',
'only_matching': True,
},
]
_SITE_MAP = {
'6abc': 'wpvi',
'abc11': 'wtvd',
'abc13': 'ktrk',
'abc30': 'kfsn',
'abc7': 'kabc',
'abc7chicago': 'wls',
'abc7news': 'kgo',
'abc7ny': 'wabc',
}
def _real_extract(self, url):
mobj = re.match(self._VALID_URL, url)
video_id = mobj.group('id')
display_id = mobj.group('display_id') or video_id
site, display_id, video_id = re.match(self._VALID_URL, url).groups()
display_id = display_id or video_id
station = self._SITE_MAP[site]
webpage = self._download_webpage(url, display_id)
data = self._download_json(
'https://api.abcotvs.com/v2/content', display_id, query={
'id': video_id,
'key': 'otv.web.%s.story' % station,
'station': station,
})['data']
video = try_get(data, lambda x: x['featuredMedia']['video'], dict) or data
video_id = compat_str(dict_get(video, ('id', 'publishedKey'), video_id))
title = video.get('title') or video['linkText']
m3u8 = self._html_search_meta(
'contentURL', webpage, 'm3u8 url', fatal=True).split('?')[0]
formats = self._extract_m3u8_formats(m3u8, display_id, 'mp4')
formats = []
m3u8_url = video.get('m3u8')
if m3u8_url:
formats = self._extract_m3u8_formats(
video['m3u8'].split('?')[0], display_id, 'mp4', m3u8_id='hls', fatal=False)
mp4_url = video.get('mp4')
if mp4_url:
formats.append({
'abr': 128,
'format_id': 'https',
'height': 360,
'url': mp4_url,
'width': 640,
})
self._sort_formats(formats)
title = self._og_search_title(webpage).strip()
description = self._og_search_description(webpage).strip()
thumbnail = self._og_search_thumbnail(webpage)
timestamp = parse_iso8601(self._search_regex(
r'<div class="meta">\s*<time class="timeago" datetime="([^"]+)">',
webpage, 'upload date', fatal=False))
uploader = self._search_regex(
r'rel="author">([^<]+)</a>',
webpage, 'uploader', default=None)
image = video.get('image') or {}
return {
'id': video_id,
'display_id': display_id,
'title': title,
'description': description,
'thumbnail': thumbnail,
'timestamp': timestamp,
'uploader': uploader,
'description': dict_get(video, ('description', 'caption'), try_get(video, lambda x: x['meta']['description'])),
'thumbnail': dict_get(image, ('source', 'dynamicSource')),
'timestamp': int_or_none(video.get('date')),
'duration': int_or_none(video.get('length')),
'formats': formats,
}

View File

@ -7,72 +7,109 @@ import functools
from .common import InfoExtractor
from ..compat import compat_str
from ..utils import (
clean_html,
float_or_none,
int_or_none,
parse_iso8601,
try_get,
unified_timestamp,
OnDemandPagedList,
)
class ACastIE(InfoExtractor):
IE_NAME = 'acast'
_VALID_URL = r'https?://(?:www\.)?acast\.com/(?P<channel>[^/]+)/(?P<id>[^/#?]+)'
_VALID_URL = r'''(?x)
https?://
(?:
(?:(?:embed|www)\.)?acast\.com/|
play\.acast\.com/s/
)
(?P<channel>[^/]+)/(?P<id>[^/#?]+)
'''
_TESTS = [{
# test with one bling
'url': 'https://www.acast.com/condenasttraveler/-where-are-you-taipei-101-taiwan',
'md5': 'ada3de5a1e3a2a381327d749854788bb',
'info_dict': {
'id': '57de3baa-4bb0-487e-9418-2692c1277a34',
'ext': 'mp3',
'title': '"Where Are You?": Taipei 101, Taiwan',
'timestamp': 1196172000,
'upload_date': '20071127',
'description': 'md5:a0b4ef3634e63866b542e5b1199a1a0e',
'duration': 211,
}
}, {
# test with multiple blings
'url': 'https://www.acast.com/sparpodcast/2.raggarmordet-rosterurdetforflutna',
'md5': '55c0097badd7095f494c99a172f86501',
'md5': '16d936099ec5ca2d5869e3a813ee8dc4',
'info_dict': {
'id': '2a92b283-1a75-4ad8-8396-499c641de0d9',
'ext': 'mp3',
'title': '2. Raggarmordet - Röster ur det förflutna',
'description': 'md5:4f81f6d8cf2e12ee21a321d8bca32db4',
'timestamp': 1477346700,
'upload_date': '20161024',
'description': 'md5:4f81f6d8cf2e12ee21a321d8bca32db4',
'duration': 2797,
'duration': 2766.602563,
'creator': 'Anton Berg & Martin Johnson',
'series': 'Spår',
'episode': '2. Raggarmordet - Röster ur det förflutna',
}
}, {
'url': 'http://embed.acast.com/adambuxton/ep.12-adam-joeschristmaspodcast2015',
'only_matching': True,
}, {
'url': 'https://play.acast.com/s/rattegangspodden/s04e09-styckmordet-i-helenelund-del-22',
'only_matching': True,
}, {
'url': 'https://play.acast.com/s/sparpodcast/2a92b283-1a75-4ad8-8396-499c641de0d9',
'only_matching': True,
}]
def _real_extract(self, url):
channel, display_id = re.match(self._VALID_URL, url).groups()
s = self._download_json(
'https://feeder.acast.com/api/v1/shows/%s/episodes/%s' % (channel, display_id),
display_id)
media_url = s['url']
if re.search(r'[0-9a-f]{8}-(?:[0-9a-f]{4}-){3}[0-9a-f]{12}', display_id):
episode_url = s.get('episodeUrl')
if episode_url:
display_id = episode_url
else:
channel, display_id = re.match(self._VALID_URL, s['link']).groups()
cast_data = self._download_json(
'https://embed.acast.com/api/acasts/%s/%s' % (channel, display_id), display_id)
'https://play-api.acast.com/splash/%s/%s' % (channel, display_id),
display_id)['result']
e = cast_data['episode']
title = e.get('name') or s['title']
return {
'id': compat_str(cast_data['id']),
'id': compat_str(e['id']),
'display_id': display_id,
'url': [b['audio'] for b in cast_data['blings'] if b['type'] == 'BlingAudio'][0],
'title': cast_data['name'],
'description': cast_data.get('description'),
'thumbnail': cast_data.get('image'),
'timestamp': parse_iso8601(cast_data.get('publishingDate')),
'duration': int_or_none(cast_data.get('duration')),
'url': media_url,
'title': title,
'description': e.get('summary') or clean_html(e.get('description') or s.get('description')),
'thumbnail': e.get('image'),
'timestamp': unified_timestamp(e.get('publishingDate') or s.get('publishDate')),
'duration': float_or_none(e.get('duration') or s.get('duration')),
'filesize': int_or_none(e.get('contentLength')),
'creator': try_get(cast_data, lambda x: x['show']['author'], compat_str),
'series': try_get(cast_data, lambda x: x['show']['name'], compat_str),
'season_number': int_or_none(e.get('seasonNumber')),
'episode': title,
'episode_number': int_or_none(e.get('episodeNumber')),
}
class ACastChannelIE(InfoExtractor):
IE_NAME = 'acast:channel'
_VALID_URL = r'https?://(?:www\.)?acast\.com/(?P<id>[^/#?]+)'
_TEST = {
'url': 'https://www.acast.com/condenasttraveler',
_VALID_URL = r'''(?x)
https?://
(?:
(?:www\.)?acast\.com/|
play\.acast\.com/s/
)
(?P<id>[^/#?]+)
'''
_TESTS = [{
'url': 'https://www.acast.com/todayinfocus',
'info_dict': {
'id': '50544219-29bb-499e-a083-6087f4cb7797',
'title': 'Condé Nast Traveler Podcast',
'description': 'md5:98646dee22a5b386626ae31866638fbd',
'id': '4efc5294-5385-4847-98bd-519799ce5786',
'title': 'Today in Focus',
'description': 'md5:9ba5564de5ce897faeb12963f4537a64',
},
'playlist_mincount': 20,
}
_API_BASE_URL = 'https://www.acast.com/api/'
'playlist_mincount': 35,
}, {
'url': 'http://play.acast.com/s/ft-banking-weekly',
'only_matching': True,
}]
_API_BASE_URL = 'https://play.acast.com/api/'
_PAGE_SIZE = 10
@classmethod
@ -85,7 +122,7 @@ class ACastChannelIE(InfoExtractor):
channel_slug, note='Download page %d of channel data' % page)
for cast in casts:
yield self.url_result(
'https://www.acast.com/%s/%s' % (channel_slug, cast['url']),
'https://play.acast.com/s/%s/%s' % (channel_slug, cast['url']),
'ACast', cast['id'])
def _real_extract(self, url):

View File

@ -1,94 +0,0 @@
from __future__ import unicode_literals
import re
from .common import InfoExtractor
from ..compat import (
compat_HTTPError,
compat_str,
compat_urllib_parse_urlencode,
compat_urllib_parse_urlparse,
)
from ..utils import (
ExtractorError,
qualities,
)
class AddAnimeIE(InfoExtractor):
_VALID_URL = r'https?://(?:\w+\.)?add-anime\.net/(?:watch_video\.php\?(?:.*?)v=|video/)(?P<id>[\w_]+)'
_TESTS = [{
'url': 'http://www.add-anime.net/watch_video.php?v=24MR3YO5SAS9',
'md5': '72954ea10bc979ab5e2eb288b21425a0',
'info_dict': {
'id': '24MR3YO5SAS9',
'ext': 'mp4',
'description': 'One Piece 606',
'title': 'One Piece 606',
}
}, {
'url': 'http://add-anime.net/video/MDUGWYKNGBD8/One-Piece-687',
'only_matching': True,
}]
def _real_extract(self, url):
video_id = self._match_id(url)
try:
webpage = self._download_webpage(url, video_id)
except ExtractorError as ee:
if not isinstance(ee.cause, compat_HTTPError) or \
ee.cause.code != 503:
raise
redir_webpage = ee.cause.read().decode('utf-8')
action = self._search_regex(
r'<form id="challenge-form" action="([^"]+)"',
redir_webpage, 'Redirect form')
vc = self._search_regex(
r'<input type="hidden" name="jschl_vc" value="([^"]+)"/>',
redir_webpage, 'redirect vc value')
av = re.search(
r'a\.value = ([0-9]+)[+]([0-9]+)[*]([0-9]+);',
redir_webpage)
if av is None:
raise ExtractorError('Cannot find redirect math task')
av_res = int(av.group(1)) + int(av.group(2)) * int(av.group(3))
parsed_url = compat_urllib_parse_urlparse(url)
av_val = av_res + len(parsed_url.netloc)
confirm_url = (
parsed_url.scheme + '://' + parsed_url.netloc +
action + '?' +
compat_urllib_parse_urlencode({
'jschl_vc': vc, 'jschl_answer': compat_str(av_val)}))
self._download_webpage(
confirm_url, video_id,
note='Confirming after redirect')
webpage = self._download_webpage(url, video_id)
FORMATS = ('normal', 'hq')
quality = qualities(FORMATS)
formats = []
for format_id in FORMATS:
rex = r"var %s_video_file = '(.*?)';" % re.escape(format_id)
video_url = self._search_regex(rex, webpage, 'video file URLx',
fatal=False)
if not video_url:
continue
formats.append({
'format_id': format_id,
'url': video_url,
'quality': quality(format_id),
})
self._sort_formats(formats)
video_title = self._og_search_title(webpage)
video_description = self._og_search_description(webpage)
return {
'_type': 'video',
'id': video_id,
'formats': formats,
'title': video_title,
'description': video_description
}

207
youtube_dl/extractor/adn.py Normal file
View File

@ -0,0 +1,207 @@
# coding: utf-8
from __future__ import unicode_literals
import base64
import binascii
import json
import os
import random
from .common import InfoExtractor
from ..aes import aes_cbc_decrypt
from ..compat import (
compat_b64decode,
compat_ord,
)
from ..utils import (
bytes_to_intlist,
bytes_to_long,
ExtractorError,
float_or_none,
intlist_to_bytes,
long_to_bytes,
pkcs1pad,
strip_or_none,
urljoin,
)
class ADNIE(InfoExtractor):
IE_DESC = 'Anime Digital Network'
_VALID_URL = r'https?://(?:www\.)?animedigitalnetwork\.fr/video/[^/]+/(?P<id>\d+)'
_TEST = {
'url': 'http://animedigitalnetwork.fr/video/blue-exorcist-kyoto-saga/7778-episode-1-debut-des-hostilites',
'md5': 'e497370d847fd79d9d4c74be55575c7a',
'info_dict': {
'id': '7778',
'ext': 'mp4',
'title': 'Blue Exorcist - Kyôto Saga - Épisode 1',
'description': 'md5:2f7b5aa76edbc1a7a92cedcda8a528d5',
}
}
_BASE_URL = 'http://animedigitalnetwork.fr'
_RSA_KEY = (0xc35ae1e4356b65a73b551493da94b8cb443491c0aa092a357a5aee57ffc14dda85326f42d716e539a34542a0d3f363adf16c5ec222d713d5997194030ee2e4f0d1fb328c01a81cf6868c090d50de8e169c6b13d1675b9eeed1cbc51e1fffca9b38af07f37abd790924cd3bee59d0257cfda4fe5f3f0534877e21ce5821447d1b, 65537)
_POS_ALIGN_MAP = {
'start': 1,
'end': 3,
}
_LINE_ALIGN_MAP = {
'middle': 8,
'end': 4,
}
@staticmethod
def _ass_subtitles_timecode(seconds):
return '%01d:%02d:%02d.%02d' % (seconds / 3600, (seconds % 3600) / 60, seconds % 60, (seconds % 1) * 100)
def _get_subtitles(self, sub_path, video_id):
if not sub_path:
return None
enc_subtitles = self._download_webpage(
urljoin(self._BASE_URL, sub_path),
video_id, 'Downloading subtitles location', fatal=False) or '{}'
subtitle_location = (self._parse_json(enc_subtitles, video_id, fatal=False) or {}).get('location')
if subtitle_location:
enc_subtitles = self._download_webpage(
urljoin(self._BASE_URL, subtitle_location),
video_id, 'Downloading subtitles data', fatal=False,
headers={'Origin': 'https://animedigitalnetwork.fr'})
if not enc_subtitles:
return None
# http://animedigitalnetwork.fr/components/com_vodvideo/videojs/adn-vjs.min.js
dec_subtitles = intlist_to_bytes(aes_cbc_decrypt(
bytes_to_intlist(compat_b64decode(enc_subtitles[24:])),
bytes_to_intlist(binascii.unhexlify(self._K + '4b8ef13ec1872730')),
bytes_to_intlist(compat_b64decode(enc_subtitles[:24]))
))
subtitles_json = self._parse_json(
dec_subtitles[:-compat_ord(dec_subtitles[-1])].decode(),
None, fatal=False)
if not subtitles_json:
return None
subtitles = {}
for sub_lang, sub in subtitles_json.items():
ssa = '''[Script Info]
ScriptType:V4.00
[V4 Styles]
Format: Name,Fontname,Fontsize,PrimaryColour,SecondaryColour,TertiaryColour,BackColour,Bold,Italic,BorderStyle,Outline,Shadow,Alignment,MarginL,MarginR,MarginV,AlphaLevel,Encoding
Style: Default,Arial,18,16777215,16777215,16777215,0,-1,0,1,1,0,2,20,20,20,0,0
[Events]
Format: Marked,Start,End,Style,Name,MarginL,MarginR,MarginV,Effect,Text'''
for current in sub:
start, end, text, line_align, position_align = (
float_or_none(current.get('startTime')),
float_or_none(current.get('endTime')),
current.get('text'), current.get('lineAlign'),
current.get('positionAlign'))
if start is None or end is None or text is None:
continue
alignment = self._POS_ALIGN_MAP.get(position_align, 2) + self._LINE_ALIGN_MAP.get(line_align, 0)
ssa += os.linesep + 'Dialogue: Marked=0,%s,%s,Default,,0,0,0,,%s%s' % (
self._ass_subtitles_timecode(start),
self._ass_subtitles_timecode(end),
'{\\a%d}' % alignment if alignment != 2 else '',
text.replace('\n', '\\N').replace('<i>', '{\\i1}').replace('</i>', '{\\i0}'))
if sub_lang == 'vostf':
sub_lang = 'fr'
subtitles.setdefault(sub_lang, []).extend([{
'ext': 'json',
'data': json.dumps(sub),
}, {
'ext': 'ssa',
'data': ssa,
}])
return subtitles
def _real_extract(self, url):
video_id = self._match_id(url)
webpage = self._download_webpage(url, video_id)
player_config = self._parse_json(self._search_regex(
r'playerConfig\s*=\s*({.+});', webpage,
'player config', default='{}'), video_id, fatal=False)
if not player_config:
config_url = urljoin(self._BASE_URL, self._search_regex(
r'(?:id="player"|class="[^"]*adn-player-container[^"]*")[^>]+data-url="([^"]+)"',
webpage, 'config url'))
player_config = self._download_json(
config_url, video_id,
'Downloading player config JSON metadata')['player']
video_info = {}
video_info_str = self._search_regex(
r'videoInfo\s*=\s*({.+});', webpage,
'video info', fatal=False)
if video_info_str:
video_info = self._parse_json(
video_info_str, video_id, fatal=False) or {}
options = player_config.get('options') or {}
metas = options.get('metas') or {}
links = player_config.get('links') or {}
sub_path = player_config.get('subtitles')
error = None
if not links:
links_url = player_config.get('linksurl') or options['videoUrl']
token = options['token']
self._K = ''.join([random.choice('0123456789abcdef') for _ in range(16)])
message = bytes_to_intlist(json.dumps({
'k': self._K,
'e': 60,
't': token,
}))
padded_message = intlist_to_bytes(pkcs1pad(message, 128))
n, e = self._RSA_KEY
encrypted_message = long_to_bytes(pow(bytes_to_long(padded_message), e, n))
authorization = base64.b64encode(encrypted_message).decode()
links_data = self._download_json(
urljoin(self._BASE_URL, links_url), video_id,
'Downloading links JSON metadata', headers={
'Authorization': 'Bearer ' + authorization,
})
links = links_data.get('links') or {}
metas = metas or links_data.get('meta') or {}
sub_path = sub_path or links_data.get('subtitles') or \
'index.php?option=com_vodapi&task=subtitles.getJSON&format=json&id=' + video_id
sub_path += '&token=' + token
error = links_data.get('error')
title = metas.get('title') or video_info['title']
formats = []
for format_id, qualities in links.items():
if not isinstance(qualities, dict):
continue
for quality, load_balancer_url in qualities.items():
load_balancer_data = self._download_json(
load_balancer_url, video_id,
'Downloading %s %s JSON metadata' % (format_id, quality),
fatal=False) or {}
m3u8_url = load_balancer_data.get('location')
if not m3u8_url:
continue
m3u8_formats = self._extract_m3u8_formats(
m3u8_url, video_id, 'mp4', 'm3u8_native',
m3u8_id=format_id, fatal=False)
if format_id == 'vf':
for f in m3u8_formats:
f['language'] = 'fr'
formats.extend(m3u8_formats)
if not error:
error = options.get('error')
if not formats and error:
raise ExtractorError('%s said: %s' % (self.IE_NAME, error), expected=True)
self._sort_formats(formats)
return {
'id': video_id,
'title': title,
'description': strip_or_none(metas.get('summary') or video_info.get('resume')),
'thumbnail': video_info.get('image'),
'formats': formats,
'subtitles': self.extract_subtitles(sub_path, video_id),
'episode': metas.get('subtitle') or video_info.get('videoTitle'),
'series': video_info.get('playlistTitle'),
}

View File

@ -0,0 +1,37 @@
# coding: utf-8
from __future__ import unicode_literals
from .common import InfoExtractor
from ..compat import (
compat_parse_qs,
compat_urlparse,
)
class AdobeConnectIE(InfoExtractor):
_VALID_URL = r'https?://\w+\.adobeconnect\.com/(?P<id>[\w-]+)'
def _real_extract(self, url):
video_id = self._match_id(url)
webpage = self._download_webpage(url, video_id)
title = self._html_search_regex(r'<title>(.+?)</title>', webpage, 'title')
qs = compat_parse_qs(self._search_regex(r"swfUrl\s*=\s*'([^']+)'", webpage, 'swf url').split('?')[1])
is_live = qs.get('isLive', ['false'])[0] == 'true'
formats = []
for con_string in qs['conStrings'][0].split(','):
formats.append({
'format_id': con_string.split('://')[0],
'app': compat_urlparse.quote('?' + con_string.split('?')[1] + 'flvplayerapp/' + qs['appInstance'][0]),
'ext': 'flv',
'play_path': 'mp4:' + qs['streamName'][0],
'rtmp_conn': 'S:' + qs['ticket'][0],
'rtmp_live': is_live,
'url': con_string,
})
return {
'id': video_id,
'title': self._live_title(title) if is_live else title,
'formats': formats,
'is_live': is_live,
}

View File

@ -6,12 +6,16 @@ import time
import xml.etree.ElementTree as etree
from .common import InfoExtractor
from ..compat import compat_urlparse
from ..compat import (
compat_kwargs,
compat_urlparse,
)
from ..utils import (
unescapeHTML,
urlencode_postdata,
unified_timestamp,
ExtractorError,
NO_DEFAULT,
)
@ -21,6 +25,16 @@ MSO_INFO = {
'username_field': 'username',
'password_field': 'password',
},
'ATT': {
'name': 'AT&T U-verse',
'username_field': 'userid',
'password_field': 'password',
},
'ATTOTT': {
'name': 'DIRECTV NOW',
'username_field': 'email',
'password_field': 'loginpassword',
},
'Rogers': {
'name': 'Rogers',
'username_field': 'UserName',
@ -31,6 +45,26 @@ MSO_INFO = {
'username_field': 'user',
'password_field': 'passwd',
},
'TWC': {
'name': 'Time Warner Cable | Spectrum',
'username_field': 'Ecom_User_ID',
'password_field': 'Ecom_Password',
},
'Brighthouse': {
'name': 'Bright House Networks | Spectrum',
'username_field': 'j_username',
'password_field': 'j_password',
},
'Charter_Direct': {
'name': 'Charter Spectrum',
'username_field': 'IDToken1',
'password_field': 'IDToken2',
},
'Verizon': {
'name': 'Verizon FiOS',
'username_field': 'IDToken1',
'password_field': 'IDToken2',
},
'thr030': {
'name': '3 Rivers Communications'
},
@ -1293,6 +1327,15 @@ class AdobePassIE(InfoExtractor):
_USER_AGENT = 'Mozilla/5.0 (X11; Linux i686; rv:47.0) Gecko/20100101 Firefox/47.0'
_MVPD_CACHE = 'ap-mvpd'
_DOWNLOADING_LOGIN_PAGE = 'Downloading Provider Login Page'
def _download_webpage_handle(self, *args, **kwargs):
headers = self.geo_verification_headers()
headers.update(kwargs.get('headers', {}))
kwargs['headers'] = headers
return super(AdobePassIE, self)._download_webpage_handle(
*args, **compat_kwargs(kwargs))
@staticmethod
def _get_mvpd_resource(provider_id, title, guid, rating):
channel = etree.Element('channel')
@ -1335,6 +1378,21 @@ class AdobePassIE(InfoExtractor):
'Use --ap-mso to specify Adobe Pass Multiple-system operator Identifier '
'and --ap-username and --ap-password or --netrc to provide account credentials.', expected=True)
def extract_redirect_url(html, url=None, fatal=False):
# TODO: eliminate code duplication with generic extractor and move
# redirection code into _download_webpage_handle
REDIRECT_REGEX = r'[0-9]{,2};\s*(?:URL|url)=\'?([^\'"]+)'
redirect_url = self._search_regex(
r'(?i)<meta\s+(?=(?:[a-z-]+="[^"]+"\s+)*http-equiv="refresh")'
r'(?:[a-z-]+="[^"]+"\s+)*?content="%s' % REDIRECT_REGEX,
html, 'meta refresh redirect',
default=NO_DEFAULT if fatal else None, fatal=fatal)
if not redirect_url:
return None
if url:
redirect_url = compat_urlparse.urljoin(url, unescapeHTML(redirect_url))
return redirect_url
mvpd_headers = {
'ap_42': 'anonymous',
'ap_11': 'Linux i686',
@ -1374,42 +1432,82 @@ class AdobePassIE(InfoExtractor):
# Comcast page flow varies by video site and whether you
# are on Comcast's network.
provider_redirect_page, urlh = provider_redirect_page_res
# Check for Comcast auto login
if 'automatically signing you in' in provider_redirect_page:
oauth_redirect_url = self._html_search_regex(
r'window\.location\s*=\s*[\'"]([^\'"]+)',
provider_redirect_page, 'oauth redirect')
# Just need to process the request. No useful data comes back
self._download_webpage(
oauth_redirect_url, video_id, 'Confirming auto login')
else:
if '<form name="signin"' in provider_redirect_page:
# already have the form, just fill it
provider_login_page_res = provider_redirect_page_res
elif 'http-equiv="refresh"' in provider_redirect_page:
# redirects to the login page
oauth_redirect_url = self._html_search_regex(
r'content="0;\s*url=([^\'"]+)',
provider_redirect_page, 'meta refresh redirect')
oauth_redirect_url = extract_redirect_url(
provider_redirect_page, fatal=True)
provider_login_page_res = self._download_webpage_handle(
oauth_redirect_url,
video_id, 'Downloading Provider Login Page')
oauth_redirect_url, video_id,
self._DOWNLOADING_LOGIN_PAGE)
else:
provider_login_page_res = post_form(
provider_redirect_page_res, 'Downloading Provider Login Page')
provider_redirect_page_res,
self._DOWNLOADING_LOGIN_PAGE)
mvpd_confirm_page_res = post_form(provider_login_page_res, 'Logging in', {
mso_info.get('username_field', 'username'): username,
mso_info.get('password_field', 'password'): password,
mvpd_confirm_page_res = post_form(
provider_login_page_res, 'Logging in', {
mso_info['username_field']: username,
mso_info['password_field']: password,
})
mvpd_confirm_page, urlh = mvpd_confirm_page_res
if '<button class="submit" value="Resume">Resume</button>' in mvpd_confirm_page:
post_form(mvpd_confirm_page_res, 'Confirming Login')
elif mso_id == 'Verizon':
# In general, if you're connecting from a Verizon-assigned IP,
# you will not actually pass your credentials.
provider_redirect_page, urlh = provider_redirect_page_res
if 'Please wait ...' in provider_redirect_page:
saml_redirect_url = self._html_search_regex(
r'self\.parent\.location=(["\'])(?P<url>.+?)\1',
provider_redirect_page,
'SAML Redirect URL', group='url')
saml_login_page = self._download_webpage(
saml_redirect_url, video_id,
'Downloading SAML Login Page')
else:
# Normal, non-Comcast flow
saml_login_page_res = post_form(
provider_redirect_page_res, 'Logging in', {
mso_info['username_field']: username,
mso_info['password_field']: password,
})
saml_login_page, urlh = saml_login_page_res
if 'Please try again.' in saml_login_page:
raise ExtractorError(
'We\'re sorry, but either the User ID or Password entered is not correct.')
saml_login_url = self._search_regex(
r'xmlHttp\.open\("POST"\s*,\s*(["\'])(?P<url>.+?)\1',
saml_login_page, 'SAML Login URL', group='url')
saml_response_json = self._download_json(
saml_login_url, video_id, 'Downloading SAML Response',
headers={'Content-Type': 'text/xml'})
self._download_webpage(
saml_response_json['targetValue'], video_id,
'Confirming Login', data=urlencode_postdata({
'SAMLResponse': saml_response_json['SAMLResponse'],
'RelayState': saml_response_json['RelayState']
}), headers={
'Content-Type': 'application/x-www-form-urlencoded'
})
else:
# Some providers (e.g. DIRECTV NOW) have another meta refresh
# based redirect that should be followed.
provider_redirect_page, urlh = provider_redirect_page_res
provider_refresh_redirect_url = extract_redirect_url(
provider_redirect_page, url=urlh.geturl())
if provider_refresh_redirect_url:
provider_redirect_page_res = self._download_webpage_handle(
provider_refresh_redirect_url, video_id,
'Downloading Provider Redirect Page (meta refresh)')
provider_login_page_res = post_form(
provider_redirect_page_res, 'Downloading Provider Login Page')
provider_redirect_page_res, self._DOWNLOADING_LOGIN_PAGE)
mvpd_confirm_page_res = post_form(provider_login_page_res, 'Logging in', {
mso_info.get('username_field', 'username'): username,
mso_info.get('password_field', 'password'): password,
@ -1448,6 +1546,8 @@ class AdobePassIE(InfoExtractor):
self._downloader.cache.store(self._MVPD_CACHE, requestor_id, {})
count += 1
continue
if '<error' in authorize:
raise ExtractorError(xml_text(authorize, 'details'), expected=True)
authz_token = unescapeHTML(xml_text(authorize, 'authzToken'))
requestor_info[guid] = authz_token
self._downloader.cache.store(self._MVPD_CACHE, requestor_id, requestor_info)

View File

@ -1,25 +1,119 @@
from __future__ import unicode_literals
import functools
import re
from .common import InfoExtractor
from ..compat import compat_str
from ..utils import (
parse_duration,
unified_strdate,
str_to_int,
int_or_none,
float_or_none,
int_or_none,
ISO639Utils,
determine_ext,
OnDemandPagedList,
parse_duration,
str_or_none,
str_to_int,
unified_strdate,
)
class AdobeTVBaseIE(InfoExtractor):
_API_BASE_URL = 'http://tv.adobe.com/api/v4/'
def _call_api(self, path, video_id, query, note=None):
return self._download_json(
'http://tv.adobe.com/api/v4/' + path,
video_id, note, query=query)['data']
def _parse_subtitles(self, video_data, url_key):
subtitles = {}
for translation in video_data.get('translations', []):
vtt_path = translation.get(url_key)
if not vtt_path:
continue
lang = translation.get('language_w3c') or ISO639Utils.long2short(translation['language_medium'])
subtitles.setdefault(lang, []).append({
'ext': 'vtt',
'url': vtt_path,
})
return subtitles
def _parse_video_data(self, video_data):
video_id = compat_str(video_data['id'])
title = video_data['title']
s3_extracted = False
formats = []
for source in video_data.get('videos', []):
source_url = source.get('url')
if not source_url:
continue
f = {
'format_id': source.get('quality_level'),
'fps': int_or_none(source.get('frame_rate')),
'height': int_or_none(source.get('height')),
'tbr': int_or_none(source.get('video_data_rate')),
'width': int_or_none(source.get('width')),
'url': source_url,
}
original_filename = source.get('original_filename')
if original_filename:
if not (f.get('height') and f.get('width')):
mobj = re.search(r'_(\d+)x(\d+)', original_filename)
if mobj:
f.update({
'height': int(mobj.group(2)),
'width': int(mobj.group(1)),
})
if original_filename.startswith('s3://') and not s3_extracted:
formats.append({
'format_id': 'original',
'preference': 1,
'url': original_filename.replace('s3://', 'https://s3.amazonaws.com/'),
})
s3_extracted = True
formats.append(f)
self._sort_formats(formats)
return {
'id': video_id,
'title': title,
'description': video_data.get('description'),
'thumbnail': video_data.get('thumbnail'),
'upload_date': unified_strdate(video_data.get('start_date')),
'duration': parse_duration(video_data.get('duration')),
'view_count': str_to_int(video_data.get('playcount')),
'formats': formats,
'subtitles': self._parse_subtitles(video_data, 'vtt'),
}
class AdobeTVEmbedIE(AdobeTVBaseIE):
IE_NAME = 'adobetv:embed'
_VALID_URL = r'https?://tv\.adobe\.com/embed/\d+/(?P<id>\d+)'
_TEST = {
'url': 'https://tv.adobe.com/embed/22/4153',
'md5': 'c8c0461bf04d54574fc2b4d07ac6783a',
'info_dict': {
'id': '4153',
'ext': 'flv',
'title': 'Creating Graphics Optimized for BlackBerry',
'description': 'md5:eac6e8dced38bdaae51cd94447927459',
'thumbnail': r're:https?://.*\.jpg$',
'upload_date': '20091109',
'duration': 377,
'view_count': int,
},
}
def _real_extract(self, url):
video_id = self._match_id(url)
video_data = self._call_api(
'episode/' + video_id, video_id, {'disclosure': 'standard'})[0]
return self._parse_video_data(video_data)
class AdobeTVIE(AdobeTVBaseIE):
IE_NAME = 'adobetv'
_VALID_URL = r'https?://tv\.adobe\.com/(?:(?P<language>fr|de|es|jp)/)?watch/(?P<show_urlname>[^/]+)/(?P<id>[^/]+)'
_TEST = {
@ -42,45 +136,33 @@ class AdobeTVIE(AdobeTVBaseIE):
if not language:
language = 'en'
video_data = self._download_json(
self._API_BASE_URL + 'episode/get/?language=%s&show_urlname=%s&urlname=%s&disclosure=standard' % (language, show_urlname, urlname),
urlname)['data'][0]
formats = [{
'url': source['url'],
'format_id': source.get('quality_level') or source['url'].split('-')[-1].split('.')[0] or None,
'width': int_or_none(source.get('width')),
'height': int_or_none(source.get('height')),
'tbr': int_or_none(source.get('video_data_rate')),
} for source in video_data['videos']]
self._sort_formats(formats)
return {
'id': compat_str(video_data['id']),
'title': video_data['title'],
'description': video_data.get('description'),
'thumbnail': video_data.get('thumbnail'),
'upload_date': unified_strdate(video_data.get('start_date')),
'duration': parse_duration(video_data.get('duration')),
'view_count': str_to_int(video_data.get('playcount')),
'formats': formats,
}
video_data = self._call_api(
'episode/get', urlname, {
'disclosure': 'standard',
'language': language,
'show_urlname': show_urlname,
'urlname': urlname,
})[0]
return self._parse_video_data(video_data)
class AdobeTVPlaylistBaseIE(AdobeTVBaseIE):
def _parse_page_data(self, page_data):
return [self.url_result(self._get_element_url(element_data)) for element_data in page_data]
_PAGE_SIZE = 25
def _extract_playlist_entries(self, url, display_id):
page = self._download_json(url, display_id)
entries = self._parse_page_data(page['data'])
for page_num in range(2, page['paging']['pages'] + 1):
entries.extend(self._parse_page_data(
self._download_json(url + '&page=%d' % page_num, display_id)['data']))
return entries
def _fetch_page(self, display_id, query, page):
page += 1
query['page'] = page
for element_data in self._call_api(
self._RESOURCE, display_id, query, 'Download Page %d' % page):
yield self._process_data(element_data)
def _extract_playlist_entries(self, display_id, query):
return OnDemandPagedList(functools.partial(
self._fetch_page, display_id, query), self._PAGE_SIZE)
class AdobeTVShowIE(AdobeTVPlaylistBaseIE):
IE_NAME = 'adobetv:show'
_VALID_URL = r'https?://tv\.adobe\.com/(?:(?P<language>fr|de|es|jp)/)?show/(?P<id>[^/]+)'
_TEST = {
@ -92,26 +174,31 @@ class AdobeTVShowIE(AdobeTVPlaylistBaseIE):
},
'playlist_mincount': 136,
}
def _get_element_url(self, element_data):
return element_data['urls'][0]
_RESOURCE = 'episode'
_process_data = AdobeTVBaseIE._parse_video_data
def _real_extract(self, url):
language, show_urlname = re.match(self._VALID_URL, url).groups()
if not language:
language = 'en'
query = 'language=%s&show_urlname=%s' % (language, show_urlname)
query = {
'disclosure': 'standard',
'language': language,
'show_urlname': show_urlname,
}
show_data = self._download_json(self._API_BASE_URL + 'show/get/?%s' % query, show_urlname)['data'][0]
show_data = self._call_api(
'show/get', show_urlname, query)[0]
return self.playlist_result(
self._extract_playlist_entries(self._API_BASE_URL + 'episode/?%s' % query, show_urlname),
compat_str(show_data['id']),
show_data['show_name'],
show_data['show_description'])
self._extract_playlist_entries(show_urlname, query),
str_or_none(show_data.get('id')),
show_data.get('show_name'),
show_data.get('show_description'))
class AdobeTVChannelIE(AdobeTVPlaylistBaseIE):
IE_NAME = 'adobetv:channel'
_VALID_URL = r'https?://tv\.adobe\.com/(?:(?P<language>fr|de|es|jp)/)?channel/(?P<id>[^/]+)(?:/(?P<category_urlname>[^/]+))?'
_TEST = {
@ -121,24 +208,30 @@ class AdobeTVChannelIE(AdobeTVPlaylistBaseIE):
},
'playlist_mincount': 96,
}
_RESOURCE = 'show'
def _get_element_url(self, element_data):
return element_data['url']
def _process_data(self, show_data):
return self.url_result(
show_data['url'], 'AdobeTVShow', str_or_none(show_data.get('id')))
def _real_extract(self, url):
language, channel_urlname, category_urlname = re.match(self._VALID_URL, url).groups()
if not language:
language = 'en'
query = 'language=%s&channel_urlname=%s' % (language, channel_urlname)
query = {
'channel_urlname': channel_urlname,
'language': language,
}
if category_urlname:
query += '&category_urlname=%s' % category_urlname
query['category_urlname'] = category_urlname
return self.playlist_result(
self._extract_playlist_entries(self._API_BASE_URL + 'show/?%s' % query, channel_urlname),
self._extract_playlist_entries(channel_urlname, query),
channel_urlname)
class AdobeTVVideoIE(InfoExtractor):
class AdobeTVVideoIE(AdobeTVBaseIE):
IE_NAME = 'adobetv:video'
_VALID_URL = r'https?://video\.tv\.adobe\.com/v/(?P<id>\d+)'
_TEST = {
@ -160,38 +253,36 @@ class AdobeTVVideoIE(InfoExtractor):
video_data = self._parse_json(self._search_regex(
r'var\s+bridge\s*=\s*([^;]+);', webpage, 'bridged data'), video_id)
title = video_data['title']
formats = [{
'format_id': '%s-%s' % (determine_ext(source['src']), source.get('height')),
'url': source['src'],
'width': int_or_none(source.get('width')),
'height': int_or_none(source.get('height')),
'tbr': int_or_none(source.get('bitrate')),
} for source in video_data['sources']]
formats = []
sources = video_data.get('sources') or []
for source in sources:
source_src = source.get('src')
if not source_src:
continue
formats.append({
'filesize': int_or_none(source.get('kilobytes') or None, invscale=1000),
'format_id': '-'.join(filter(None, [source.get('format'), source.get('label')])),
'height': int_or_none(source.get('height') or None),
'tbr': int_or_none(source.get('bitrate') or None),
'width': int_or_none(source.get('width') or None),
'url': source_src,
})
self._sort_formats(formats)
# For both metadata and downloaded files the duration varies among
# formats. I just pick the max one
duration = max(filter(None, [
float_or_none(source.get('duration'), scale=1000)
for source in video_data['sources']]))
subtitles = {}
for translation in video_data.get('translations', []):
lang_id = translation.get('language_w3c') or ISO639Utils.long2short(translation['language_medium'])
if lang_id not in subtitles:
subtitles[lang_id] = []
subtitles[lang_id].append({
'url': translation['vttPath'],
'ext': 'vtt',
})
for source in sources]))
return {
'id': video_id,
'formats': formats,
'title': video_data['title'],
'title': title,
'description': video_data.get('description'),
'thumbnail': video_data['video'].get('poster'),
'thumbnail': video_data.get('video', {}).get('poster'),
'duration': duration,
'subtitles': subtitles,
'subtitles': self._parse_subtitles(video_data, 'vttPath'),
}

View File

@ -1,95 +1,64 @@
# coding: utf-8
from __future__ import unicode_literals
import json
import re
from .turner import TurnerBaseIE
from ..utils import (
ExtractorError,
determine_ext,
float_or_none,
int_or_none,
mimetype2ext,
parse_age_limit,
parse_iso8601,
strip_or_none,
try_get,
)
class AdultSwimIE(TurnerBaseIE):
_VALID_URL = r'https?://(?:www\.)?adultswim\.com/videos/(?P<is_playlist>playlists/)?(?P<show_path>[^/]+)/(?P<episode_path>[^/?#]+)/?'
_VALID_URL = r'https?://(?:www\.)?adultswim\.com/videos/(?P<show_path>[^/?#]+)(?:/(?P<episode_path>[^/?#]+))?'
_TESTS = [{
'url': 'http://adultswim.com/videos/rick-and-morty/pilot',
'playlist': [
{
'md5': '247572debc75c7652f253c8daa51a14d',
'info_dict': {
'id': 'rQxZvXQ4ROaSOqq-or2Mow-0',
'ext': 'flv',
'title': 'Rick and Morty - Pilot Part 1',
'description': "Rick moves in with his daughter's family and establishes himself as a bad influence on his grandson, Morty. "
},
},
{
'md5': '77b0e037a4b20ec6b98671c4c379f48d',
'info_dict': {
'id': 'rQxZvXQ4ROaSOqq-or2Mow-3',
'ext': 'flv',
'title': 'Rick and Morty - Pilot Part 4',
'description': "Rick moves in with his daughter's family and establishes himself as a bad influence on his grandson, Morty. "
},
},
],
'info_dict': {
'id': 'rQxZvXQ4ROaSOqq-or2Mow',
'title': 'Rick and Morty - Pilot',
'description': "Rick moves in with his daughter's family and establishes himself as a bad influence on his grandson, Morty. "
},
'skip': 'This video is only available for registered users',
}, {
'url': 'http://www.adultswim.com/videos/playlists/american-parenting/putting-francine-out-of-business/',
'playlist': [
{
'md5': '2eb5c06d0f9a1539da3718d897f13ec5',
'info_dict': {
'id': '-t8CamQlQ2aYZ49ItZCFog-0',
'ext': 'flv',
'title': 'American Dad - Putting Francine Out of Business',
'description': 'Stan hatches a plan to get Francine out of the real estate business.Watch more American Dad on [adult swim].'
},
}
],
'info_dict': {
'id': '-t8CamQlQ2aYZ49ItZCFog',
'title': 'American Dad - Putting Francine Out of Business',
'description': 'Stan hatches a plan to get Francine out of the real estate business.Watch more American Dad on [adult swim].'
},
}, {
'url': 'http://www.adultswim.com/videos/tim-and-eric-awesome-show-great-job/dr-steve-brule-for-your-wine/',
'playlist': [
{
'md5': '3e346a2ab0087d687a05e1e7f3b3e529',
'info_dict': {
'id': 'sY3cMUR_TbuE4YmdjzbIcQ-0',
'ext': 'mp4',
'title': 'Tim and Eric Awesome Show Great Job! - Dr. Steve Brule, For Your Wine',
'description': 'Dr. Brule reports live from Wine Country with a special report on wines. \r\nWatch Tim and Eric Awesome Show Great Job! episode #20, "Embarrassed" on Adult Swim.\r\n\r\n',
},
}
],
'info_dict': {
'id': 'sY3cMUR_TbuE4YmdjzbIcQ',
'title': 'Tim and Eric Awesome Show Great Job! - Dr. Steve Brule, For Your Wine',
'description': 'Dr. Brule reports live from Wine Country with a special report on wines. \r\nWatch Tim and Eric Awesome Show Great Job! episode #20, "Embarrassed" on Adult Swim.\r\n\r\n',
'title': 'Rick and Morty - Pilot',
'description': 'Rick moves in with his daughter\'s family and establishes himself as a bad influence on his grandson, Morty.',
'timestamp': 1543294800,
'upload_date': '20181127',
},
'params': {
# m3u8 download
'skip_download': True,
}
},
'expected_warnings': ['Unable to download f4m manifest'],
}, {
'url': 'http://www.adultswim.com/videos/tim-and-eric-awesome-show-great-job/dr-steve-brule-for-your-wine/',
'info_dict': {
'id': 'sY3cMUR_TbuE4YmdjzbIcQ',
'ext': 'mp4',
'title': 'Tim and Eric Awesome Show Great Job! - Dr. Steve Brule, For Your Wine',
'description': 'Dr. Brule reports live from Wine Country with a special report on wines. \nWatch Tim and Eric Awesome Show Great Job! episode #20, "Embarrassed" on Adult Swim.',
'upload_date': '20080124',
'timestamp': 1201150800,
},
'params': {
# m3u8 download
'skip_download': True,
},
'skip': '404 Not Found',
}, {
# heroMetadata.trailer
'url': 'http://www.adultswim.com/videos/decker/inside-decker-a-new-hero/',
'info_dict': {
'id': 'I0LQFQkaSUaFp8PnAWHhoQ',
'ext': 'mp4',
'title': 'Decker - Inside Decker: A New Hero',
'description': 'md5:c916df071d425d62d70c86d4399d3ee0',
'duration': 249.008,
'description': 'The guys recap the conclusion of the season. They announce a new hero, take a peek into the Victorville Film Archive and welcome back the talented James Dean.',
'timestamp': 1469480460,
'upload_date': '20160725',
},
'params': {
# m3u8 download
@ -97,136 +66,137 @@ class AdultSwimIE(TurnerBaseIE):
},
'expected_warnings': ['Unable to download f4m manifest'],
}, {
'url': 'http://www.adultswim.com/videos/toonami/friday-october-14th-2016/',
'url': 'http://www.adultswim.com/videos/attack-on-titan',
'info_dict': {
'id': 'eYiLsKVgQ6qTC6agD67Sig',
'title': 'Toonami - Friday, October 14th, 2016',
'description': 'md5:99892c96ffc85e159a428de85c30acde',
'id': 'attack-on-titan',
'title': 'Attack on Titan',
'description': 'md5:41caa9416906d90711e31dc00cb7db7e',
},
'playlist': [{
'md5': '',
'playlist_mincount': 12,
}, {
'url': 'http://www.adultswim.com/videos/streams/williams-stream',
'info_dict': {
'id': 'eYiLsKVgQ6qTC6agD67Sig',
'id': 'd8DEBj7QRfetLsRgFnGEyg',
'ext': 'mp4',
'title': 'Toonami - Friday, October 14th, 2016',
'description': 'md5:99892c96ffc85e159a428de85c30acde',
'title': r're:^Williams Stream \d{4}-\d{2}-\d{2} \d{2}:\d{2}$',
'description': 'original programming',
},
}],
'params': {
# m3u8 download
'skip_download': True,
},
'expected_warnings': ['Unable to download f4m manifest'],
'skip': '404 Not Found',
}]
@staticmethod
def find_video_info(collection, slug):
for video in collection.get('videos'):
if video.get('slug') == slug:
return video
@staticmethod
def find_collection_by_linkURL(collections, linkURL):
for collection in collections:
if collection.get('linkURL') == linkURL:
return collection
@staticmethod
def find_collection_containing_video(collections, slug):
for collection in collections:
for video in collection.get('videos'):
if video.get('slug') == slug:
return collection, video
return None, None
def _real_extract(self, url):
mobj = re.match(self._VALID_URL, url)
show_path = mobj.group('show_path')
episode_path = mobj.group('episode_path')
is_playlist = True if mobj.group('is_playlist') else False
webpage = self._download_webpage(url, episode_path)
# Extract the value of `bootstrappedData` from the Javascript in the page.
bootstrapped_data = self._parse_json(self._search_regex(
r'var bootstrappedData = ({.*});', webpage, 'bootstraped data'), episode_path)
# Downloading videos from a /videos/playlist/ URL needs to be handled differently.
# NOTE: We are only downloading one video (the current one) not the playlist
if is_playlist:
collections = bootstrapped_data['playlists']['collections']
collection = self.find_collection_by_linkURL(collections, show_path)
video_info = self.find_video_info(collection, episode_path)
show_title = video_info['showTitle']
segment_ids = [video_info['videoPlaybackID']]
else:
collections = bootstrapped_data['show']['collections']
collection, video_info = self.find_collection_containing_video(collections, episode_path)
# Video wasn't found in the collections, let's try `slugged_video`.
if video_info is None:
if bootstrapped_data.get('slugged_video', {}).get('slug') == episode_path:
video_info = bootstrapped_data['slugged_video']
if not video_info:
video_info = bootstrapped_data.get(
'heroMetadata', {}).get('trailer', {}).get('video')
if not video_info:
video_info = bootstrapped_data.get('onlineOriginals', [None])[0]
if not video_info:
raise ExtractorError('Unable to find video info')
show = bootstrapped_data['show']
show_title = show['title']
stream = video_info.get('stream')
if stream and stream.get('videoPlaybackID'):
segment_ids = [stream['videoPlaybackID']]
elif video_info.get('clips'):
segment_ids = [clip['videoPlaybackID'] for clip in video_info['clips']]
elif video_info.get('videoPlaybackID'):
segment_ids = [video_info['videoPlaybackID']]
elif video_info.get('id'):
segment_ids = [video_info['id']]
else:
if video_info.get('auth') is True:
raise ExtractorError(
'This video is only available via cable service provider subscription that'
' is not currently supported. You may want to use --cookies.', expected=True)
else:
raise ExtractorError('Unable to find stream or clips')
episode_id = video_info['id']
episode_title = video_info['title']
episode_description = video_info.get('description')
episode_duration = int_or_none(video_info.get('duration'))
view_count = int_or_none(video_info.get('views'))
entries = []
for part_num, segment_id in enumerate(segment_ids):
segement_info = self._extract_cvp_info(
'http://www.adultswim.com/videos/api/v0/assets?id=%s&platform=desktop' % segment_id,
segment_id, {
'secure': {
'media_src': 'http://androidhls-secure.cdn.turner.com/adultswim/big',
'tokenizer_src': 'http://www.adultswim.com/astv/mvpd/processors/services/token_ipadAdobe.do',
},
})
segment_title = '%s - %s' % (show_title, episode_title)
if len(segment_ids) > 1:
segment_title += ' Part %d' % (part_num + 1)
segement_info.update({
'id': segment_id,
'title': segment_title,
'description': episode_description,
})
entries.append(segement_info)
return {
'_type': 'playlist',
'id': episode_id,
'display_id': episode_path,
'entries': entries,
'title': '%s - %s' % (show_title, episode_title),
'description': episode_description,
'duration': episode_duration,
'view_count': view_count,
show_path, episode_path = re.match(self._VALID_URL, url).groups()
display_id = episode_path or show_path
query = '''query {
getShowBySlug(slug:"%s") {
%%s
}
}''' % show_path
if episode_path:
query = query % '''title
getVideoBySlug(slug:"%s") {
_id
auth
description
duration
episodeNumber
launchDate
mediaID
seasonNumber
poster
title
tvRating
}''' % episode_path
['getVideoBySlug']
else:
query = query % '''metaDescription
title
videos(first:1000,sort:["episode_number"]) {
edges {
node {
_id
slug
}
}
}'''
show_data = self._download_json(
'https://www.adultswim.com/api/search', display_id,
data=json.dumps({'query': query}).encode(),
headers={'Content-Type': 'application/json'})['data']['getShowBySlug']
if episode_path:
video_data = show_data['getVideoBySlug']
video_id = video_data['_id']
episode_title = title = video_data['title']
series = show_data.get('title')
if series:
title = '%s - %s' % (series, title)
info = {
'id': video_id,
'title': title,
'description': strip_or_none(video_data.get('description')),
'duration': float_or_none(video_data.get('duration')),
'formats': [],
'subtitles': {},
'age_limit': parse_age_limit(video_data.get('tvRating')),
'thumbnail': video_data.get('poster'),
'timestamp': parse_iso8601(video_data.get('launchDate')),
'series': series,
'season_number': int_or_none(video_data.get('seasonNumber')),
'episode': episode_title,
'episode_number': int_or_none(video_data.get('episodeNumber')),
}
auth = video_data.get('auth')
media_id = video_data.get('mediaID')
if media_id:
info.update(self._extract_ngtv_info(media_id, {
# CDN_TOKEN_APP_ID from:
# https://d2gg02c3xr550i.cloudfront.net/assets/asvp.e9c8bef24322d060ef87.bundle.js
'appId': 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhcHBJZCI6ImFzLXR2ZS1kZXNrdG9wLXB0enQ2bSIsInByb2R1Y3QiOiJ0dmUiLCJuZXR3b3JrIjoiYXMiLCJwbGF0Zm9ybSI6ImRlc2t0b3AiLCJpYXQiOjE1MzI3MDIyNzl9.BzSCk-WYOZ2GMCIaeVb8zWnzhlgnXuJTCu0jGp_VaZE',
}, {
'url': url,
'site_name': 'AdultSwim',
'auth_required': auth,
}))
if not auth:
extract_data = self._download_json(
'https://www.adultswim.com/api/shows/v1/videos/' + video_id,
video_id, query={'fields': 'stream'}, fatal=False) or {}
assets = try_get(extract_data, lambda x: x['data']['video']['stream']['assets'], list) or []
for asset in assets:
asset_url = asset.get('url')
if not asset_url:
continue
ext = determine_ext(asset_url, mimetype2ext(asset.get('mime_type')))
if ext == 'm3u8':
info['formats'].extend(self._extract_m3u8_formats(
asset_url, video_id, 'mp4', m3u8_id='hls', fatal=False))
elif ext == 'f4m':
continue
# info['formats'].extend(self._extract_f4m_formats(
# asset_url, video_id, f4m_id='hds', fatal=False))
elif ext in ('scc', 'ttml', 'vtt'):
info['subtitles'].setdefault('en', []).append({
'url': asset_url,
})
self._sort_formats(info['formats'])
return info
else:
entries = []
for edge in show_data.get('videos', {}).get('edges', []):
video = edge.get('node') or {}
slug = video.get('slug')
if not slug:
continue
entries.append(self.url_result(
'http://adultswim.com/videos/%s/%s' % (show_path, slug),
'AdultSwim', video.get('_id')))
return self.playlist_result(
entries, show_path, show_data.get('title'),
strip_or_none(show_data.get('metaDescription')))

View File

@ -1,14 +1,15 @@
# coding: utf-8
from __future__ import unicode_literals
import re
from .theplatform import ThePlatformIE
from ..utils import (
extract_attributes,
ExtractorError,
int_or_none,
smuggle_url,
update_url_query,
unescapeHTML,
extract_attributes,
get_element_by_attribute,
)
from ..compat import (
compat_urlparse,
@ -19,23 +20,76 @@ class AENetworksBaseIE(ThePlatformIE):
_THEPLATFORM_KEY = 'crazyjava'
_THEPLATFORM_SECRET = 's3cr3t'
def _extract_aen_smil(self, smil_url, video_id, auth=None):
query = {'mbr': 'true'}
if auth:
query['auth'] = auth
TP_SMIL_QUERY = [{
'assetTypes': 'high_video_ak',
'switch': 'hls_high_ak'
}, {
'assetTypes': 'high_video_s3'
}, {
'assetTypes': 'high_video_s3',
'switch': 'hls_ingest_fastly'
}]
formats = []
subtitles = {}
last_e = None
for q in TP_SMIL_QUERY:
q.update(query)
m_url = update_url_query(smil_url, q)
m_url = self._sign_url(m_url, self._THEPLATFORM_KEY, self._THEPLATFORM_SECRET)
try:
tp_formats, tp_subtitles = self._extract_theplatform_smil(
m_url, video_id, 'Downloading %s SMIL data' % (q.get('switch') or q['assetTypes']))
except ExtractorError as e:
last_e = e
continue
formats.extend(tp_formats)
subtitles = self._merge_subtitles(subtitles, tp_subtitles)
if last_e and not formats:
raise last_e
self._sort_formats(formats)
return {
'id': video_id,
'formats': formats,
'subtitles': subtitles,
}
class AENetworksIE(AENetworksBaseIE):
IE_NAME = 'aenetworks'
IE_DESC = 'A+E Networks: A&E, Lifetime, History.com, FYI Network'
_VALID_URL = r'https?://(?:www\.)?(?P<domain>(?:history|aetv|mylifetime|lifetimemovieclub)\.com|fyi\.tv)/(?:shows/(?P<show_path>[^/]+(?:/[^/]+){0,2})|movies/(?P<movie_display_id>[^/]+)(?:/full-movie)?)'
IE_DESC = 'A+E Networks: A&E, Lifetime, History.com, FYI Network and History Vault'
_VALID_URL = r'''(?x)
https?://
(?:www\.)?
(?P<domain>
(?:history(?:vault)?|aetv|mylifetime|lifetimemovieclub)\.com|
fyi\.tv
)/
(?:
shows/(?P<show_path>[^/]+(?:/[^/]+){0,2})|
movies/(?P<movie_display_id>[^/]+)(?:/full-movie)?|
specials/(?P<special_display_id>[^/]+)/(?:full-special|preview-)|
collections/[^/]+/(?P<collection_display_id>[^/]+)
)
'''
_TESTS = [{
'url': 'http://www.history.com/shows/mountain-men/season-1/episode-1',
'md5': 'a97a65f7e823ae10e9244bc5433d5fe6',
'info_dict': {
'id': '22253814',
'ext': 'mp4',
'title': 'Winter Is Coming',
'title': 'Winter is Coming',
'description': 'md5:641f424b7a19d8e24f26dea22cf59d74',
'timestamp': 1338306241,
'upload_date': '20120529',
'uploader': 'AENE-NEW',
},
'params': {
# m3u8 download
'skip_download': True,
},
'add_ie': ['ThePlatform'],
}, {
'url': 'http://www.history.com/shows/ancient-aliens/season-1',
@ -65,6 +119,15 @@ class AENetworksIE(AENetworksBaseIE):
}, {
'url': 'https://www.lifetimemovieclub.com/movies/a-killer-among-us',
'only_matching': True
}, {
'url': 'http://www.history.com/specials/sniper-into-the-kill-zone/full-special',
'only_matching': True
}, {
'url': 'https://www.historyvault.com/collections/america-the-story-of-us/westward',
'only_matching': True
}, {
'url': 'https://www.aetv.com/specials/hunting-jonbenets-killer-the-untold-story/preview-hunting-jonbenets-killer-the-untold-story',
'only_matching': True
}]
_DOMAIN_TO_REQUESTOR_ID = {
'history.com': 'HISTORY',
@ -75,9 +138,9 @@ class AENetworksIE(AENetworksBaseIE):
}
def _real_extract(self, url):
domain, show_path, movie_display_id = re.match(self._VALID_URL, url).groups()
display_id = show_path or movie_display_id
webpage = self._download_webpage(url, display_id)
domain, show_path, movie_display_id, special_display_id, collection_display_id = re.match(self._VALID_URL, url).groups()
display_id = show_path or movie_display_id or special_display_id or collection_display_id
webpage = self._download_webpage(url, display_id, headers=self.geo_verification_headers())
if show_path:
url_parts = show_path.split('/')
url_parts_len = len(url_parts)
@ -86,10 +149,14 @@ class AENetworksIE(AENetworksBaseIE):
for season_url_path in re.findall(r'(?s)<li[^>]+data-href="(/shows/%s/season-\d+)"' % url_parts[0], webpage):
entries.append(self.url_result(
compat_urlparse.urljoin(url, season_url_path), 'AENetworks'))
if entries:
return self.playlist_result(
entries, self._html_search_meta('aetn:SeriesId', webpage),
self._html_search_meta('aetn:SeriesTitle', webpage))
elif url_parts_len == 2:
else:
# single season
url_parts_len = 2
if url_parts_len == 2:
entries = []
for episode_item in re.findall(r'(?s)<[^>]+class="[^"]*(?:episode|program)-item[^"]*"[^>]*>', webpage):
episode_attributes = extract_attributes(episode_item)
@ -97,78 +164,52 @@ class AENetworksIE(AENetworksBaseIE):
url, episode_attributes['data-canonical'])
entries.append(self.url_result(
episode_url, 'AENetworks',
episode_attributes['data-videoid']))
episode_attributes.get('data-videoid') or episode_attributes.get('data-video-id')))
return self.playlist_result(
entries, self._html_search_meta('aetn:SeasonId', webpage))
query = {
'mbr': 'true',
'assetTypes': 'high_video_s3'
}
video_id = self._html_search_meta('aetn:VideoID', webpage)
media_url = self._search_regex(
r"media_url\s*=\s*'([^']+)'", webpage, 'video url')
[r"media_url\s*=\s*'(?P<url>[^']+)'",
r'data-media-url=(?P<url>(?:https?:)?//[^\s>]+)',
r'data-media-url=(["\'])(?P<url>(?:(?!\1).)+?)\1'],
webpage, 'video url', group='url')
theplatform_metadata = self._download_theplatform_metadata(self._search_regex(
r'https?://link.theplatform.com/s/([^?]+)', media_url, 'theplatform_path'), video_id)
r'https?://link\.theplatform\.com/s/([^?]+)', media_url, 'theplatform_path'), video_id)
info = self._parse_theplatform_metadata(theplatform_metadata)
auth = None
if theplatform_metadata.get('AETN$isBehindWall'):
requestor_id = self._DOMAIN_TO_REQUESTOR_ID[domain]
resource = self._get_mvpd_resource(
requestor_id, theplatform_metadata['title'],
theplatform_metadata.get('AETN$PPL_pplProgramId') or theplatform_metadata.get('AETN$PPL_pplProgramId_OLD'),
theplatform_metadata['ratings'][0]['rating'])
query['auth'] = self._extract_mvpd_auth(
auth = self._extract_mvpd_auth(
url, video_id, requestor_id, resource)
info.update(self._search_json_ld(webpage, video_id, fatal=False))
media_url = update_url_query(media_url, query)
media_url = self._sign_url(media_url, self._THEPLATFORM_KEY, self._THEPLATFORM_SECRET)
formats, subtitles = self._extract_theplatform_smil(media_url, video_id)
self._sort_formats(formats)
info.update({
'id': video_id,
'formats': formats,
'subtitles': subtitles,
})
info.update(self._extract_aen_smil(media_url, video_id, auth))
return info
class HistoryTopicIE(AENetworksBaseIE):
IE_NAME = 'history:topic'
IE_DESC = 'History.com Topic'
_VALID_URL = r'https?://(?:www\.)?history\.com/topics/(?:[^/]+/)?(?P<topic_id>[^/]+)(?:/[^/]+(?:/(?P<video_display_id>[^/?#]+))?)?'
_VALID_URL = r'https?://(?:www\.)?history\.com/topics/[^/]+/(?P<id>[\w+-]+?)-video'
_TESTS = [{
'url': 'http://www.history.com/topics/valentines-day/history-of-valentines-day/videos/bet-you-didnt-know-valentines-day?m=528e394da93ae&s=undefined&f=1&free=false',
'url': 'https://www.history.com/topics/valentines-day/history-of-valentines-day-video',
'info_dict': {
'id': '40700995724',
'ext': 'mp4',
'title': "Bet You Didn't Know: Valentine's Day",
'title': "History of Valentines Day",
'description': 'md5:7b57ea4829b391995b405fa60bd7b5f7',
'timestamp': 1375819729,
'upload_date': '20130806',
'uploader': 'AENE-NEW',
},
'params': {
# m3u8 download
'skip_download': True,
},
'add_ie': ['ThePlatform'],
}, {
'url': 'http://www.history.com/topics/world-war-i/world-war-i-history/videos',
'info_dict':
{
'id': 'world-war-i-history',
'title': 'World War I History',
},
'playlist_mincount': 23,
}, {
'url': 'http://www.history.com/topics/world-war-i-history/videos',
'only_matching': True,
}, {
'url': 'http://www.history.com/topics/world-war-i/world-war-i-history',
'only_matching': True,
}, {
'url': 'http://www.history.com/topics/world-war-i/world-war-i-history/speeches',
'only_matching': True,
}]
def theplatform_url_result(self, theplatform_url, video_id, query):
@ -188,27 +229,19 @@ class HistoryTopicIE(AENetworksBaseIE):
}
def _real_extract(self, url):
topic_id, video_display_id = re.match(self._VALID_URL, url).groups()
if video_display_id:
webpage = self._download_webpage(url, video_display_id)
release_url, video_id = re.search(r"_videoPlayer.play\('([^']+)'\s*,\s*'[^']+'\s*,\s*'(\d+)'\)", webpage).groups()
release_url = unescapeHTML(release_url)
return self.theplatform_url_result(
release_url, video_id, {
'mbr': 'true',
'switch': 'hls',
'assetTypes': 'high_video_ak',
display_id = self._match_id(url)
webpage = self._download_webpage(url, display_id)
video_id = self._search_regex(
r'<phoenix-iframe[^>]+src="[^"]+\btpid=(\d+)', webpage, 'tpid')
result = self._download_json(
'https://feeds.video.aetnd.com/api/v2/history/videos',
video_id, query={'filter[id]': video_id})['results'][0]
title = result['title']
info = self._extract_aen_smil(result['publicUrl'], video_id)
info.update({
'title': title,
'description': result.get('description'),
'duration': int_or_none(result.get('duration')),
'timestamp': int_or_none(result.get('added'), 1000),
})
else:
webpage = self._download_webpage(url, topic_id)
entries = []
for episode_item in re.findall(r'<a.+?data-release-url="[^"]+"[^>]*>', webpage):
video_attributes = extract_attributes(episode_item)
entries.append(self.theplatform_url_result(
video_attributes['data-release-url'], video_attributes['data-id'], {
'mbr': 'true',
'switch': 'hls',
'assetTypes': 'high_video_ak',
}))
return self.playlist_result(entries, topic_id, get_element_by_attribute('class', 'show-title', webpage))
return info

View File

@ -4,15 +4,13 @@ from __future__ import unicode_literals
import re
from .common import InfoExtractor
from ..compat import (
compat_urllib_parse_urlparse,
compat_urlparse,
)
from ..compat import compat_xpath
from ..utils import (
determine_ext,
ExtractorError,
int_or_none,
update_url_query,
xpath_element,
url_or_none,
urlencode_postdata,
xpath_text,
)
@ -32,6 +30,7 @@ class AfreecaTVIE(InfoExtractor):
)
(?P<id>\d+)
'''
_NETRC_MACHINE = 'afreecatv'
_TESTS = [{
'url': 'http://live.afreecatv.com:8079/app/index.cgi?szType=read_ucc_bbs&szBjId=dailyapril&nStationNo=16711924&nBbsNo=18605867&nTitleNo=36164052&szSkin=',
'md5': 'f72c89fe7ecc14c1b5ce506c4996046e',
@ -43,7 +42,8 @@ class AfreecaTVIE(InfoExtractor):
'uploader': 'dailyapril',
'uploader_id': 'dailyapril',
'upload_date': '20160503',
}
},
'skip': 'Video is gone',
}, {
'url': 'http://afbbs.afreecatv.com:8080/app/read_ucc_bbs.cgi?nStationNo=16711924&nTitleNo=36153164&szBjId=dailyapril&nBbsNo=18605867',
'info_dict': {
@ -71,6 +71,93 @@ class AfreecaTVIE(InfoExtractor):
'upload_date': '20160502',
},
}],
'skip': 'Video is gone',
}, {
'url': 'http://vod.afreecatv.com/PLAYER/STATION/18650793',
'info_dict': {
'id': '18650793',
'ext': 'mp4',
'title': '오늘은 다르다! 쏘님의 우월한 위아래~ 댄스리액션!',
'thumbnail': r're:^https?://.*\.jpg$',
'uploader': '윈아디',
'uploader_id': 'badkids',
'duration': 107,
},
'params': {
'skip_download': True,
},
}, {
'url': 'http://vod.afreecatv.com/PLAYER/STATION/10481652',
'info_dict': {
'id': '10481652',
'title': "BJ유트루와 함께하는 '팅커벨 메이크업!'",
'thumbnail': 're:^https?://(?:video|st)img.afreecatv.com/.*$',
'uploader': 'dailyapril',
'uploader_id': 'dailyapril',
'duration': 6492,
},
'playlist_count': 2,
'playlist': [{
'md5': 'd8b7c174568da61d774ef0203159bf97',
'info_dict': {
'id': '20160502_c4c62b9d_174361386_1',
'ext': 'mp4',
'title': "BJ유트루와 함께하는 '팅커벨 메이크업!' (part 1)",
'thumbnail': 're:^https?://(?:video|st)img.afreecatv.com/.*$',
'uploader': 'dailyapril',
'uploader_id': 'dailyapril',
'upload_date': '20160502',
'duration': 3601,
},
}, {
'md5': '58f2ce7f6044e34439ab2d50612ab02b',
'info_dict': {
'id': '20160502_39e739bb_174361386_2',
'ext': 'mp4',
'title': "BJ유트루와 함께하는 '팅커벨 메이크업!' (part 2)",
'thumbnail': 're:^https?://(?:video|st)img.afreecatv.com/.*$',
'uploader': 'dailyapril',
'uploader_id': 'dailyapril',
'upload_date': '20160502',
'duration': 2891,
},
}],
'params': {
'skip_download': True,
},
}, {
# non standard key
'url': 'http://vod.afreecatv.com/PLAYER/STATION/20515605',
'info_dict': {
'id': '20170411_BE689A0E_190960999_1_2_h',
'ext': 'mp4',
'title': '혼자사는여자집',
'thumbnail': 're:^https?://(?:video|st)img.afreecatv.com/.*$',
'uploader': '♥이슬이',
'uploader_id': 'dasl8121',
'upload_date': '20170411',
'duration': 213,
},
'params': {
'skip_download': True,
},
}, {
# PARTIAL_ADULT
'url': 'http://vod.afreecatv.com/PLAYER/STATION/32028439',
'info_dict': {
'id': '20180327_27901457_202289533_1',
'ext': 'mp4',
'title': '[생]빨개요♥ (part 1)',
'thumbnail': 're:^https?://(?:video|st)img.afreecatv.com/.*$',
'uploader': '[SA]서아',
'uploader_id': 'bjdyrksu',
'upload_date': '20180327',
'duration': 3601,
},
'params': {
'skip_download': True,
},
'expected_warnings': ['adult content'],
}, {
'url': 'http://www.afreecatv.com/player/Player.swf?szType=szBjId=djleegoon&nStationNo=11273158&nBbsNo=13161095&nTitleNo=36327652',
'only_matching': True,
@ -85,43 +172,176 @@ class AfreecaTVIE(InfoExtractor):
m = re.match(r'^(?P<upload_date>\d{8})_\w+_(?P<part>\d+)$', key)
if m:
video_key['upload_date'] = m.group('upload_date')
video_key['part'] = m.group('part')
video_key['part'] = int(m.group('part'))
return video_key
def _real_initialize(self):
self._login()
def _login(self):
username, password = self._get_login_info()
if username is None:
return
login_form = {
'szWork': 'login',
'szType': 'json',
'szUid': username,
'szPassword': password,
'isSaveId': 'false',
'szScriptVar': 'oLoginRet',
'szAction': '',
}
response = self._download_json(
'https://login.afreecatv.com/app/LoginAction.php', None,
'Logging in', data=urlencode_postdata(login_form))
_ERRORS = {
-4: 'Your account has been suspended due to a violation of our terms and policies.',
-5: 'https://member.afreecatv.com/app/user_delete_progress.php',
-6: 'https://login.afreecatv.com/membership/changeMember.php',
-8: "Hello! AfreecaTV here.\nThe username you have entered belongs to \n an account that requires a legal guardian's consent. \nIf you wish to use our services without restriction, \nplease make sure to go through the necessary verification process.",
-9: 'https://member.afreecatv.com/app/pop_login_block.php',
-11: 'https://login.afreecatv.com/afreeca/second_login.php',
-12: 'https://member.afreecatv.com/app/user_security.php',
0: 'The username does not exist or you have entered the wrong password.',
-1: 'The username does not exist or you have entered the wrong password.',
-3: 'You have entered your username/password incorrectly.',
-7: 'You cannot use your Global AfreecaTV account to access Korean AfreecaTV.',
-10: 'Sorry for the inconvenience. \nYour account has been blocked due to an unauthorized access. \nPlease contact our Help Center for assistance.',
-32008: 'You have failed to log in. Please contact our Help Center.',
}
result = int_or_none(response.get('RESULT'))
if result != 1:
error = _ERRORS.get(result, 'You have failed to log in.')
raise ExtractorError(
'Unable to login: %s said: %s' % (self.IE_NAME, error),
expected=True)
def _real_extract(self, url):
video_id = self._match_id(url)
parsed_url = compat_urllib_parse_urlparse(url)
info_url = compat_urlparse.urlunparse(parsed_url._replace(
netloc='afbbs.afreecatv.com:8080',
path='/api/video/get_video_info.php'))
webpage = self._download_webpage(url, video_id)
if re.search(r'alert\(["\']This video has been deleted', webpage):
raise ExtractorError(
'Video %s has been deleted' % video_id, expected=True)
station_id = self._search_regex(
r'nStationNo\s*=\s*(\d+)', webpage, 'station')
bbs_id = self._search_regex(
r'nBbsNo\s*=\s*(\d+)', webpage, 'bbs')
video_id = self._search_regex(
r'nTitleNo\s*=\s*(\d+)', webpage, 'title', default=video_id)
partial_view = False
for _ in range(2):
query = {
'nTitleNo': video_id,
'nStationNo': station_id,
'nBbsNo': bbs_id,
}
if partial_view:
query['partialView'] = 'SKIP_ADULT'
video_xml = self._download_xml(
update_url_query(info_url, {'nTitleNo': video_id}), video_id)
'http://afbbs.afreecatv.com:8080/api/video/get_video_info.php',
video_id, 'Downloading video info XML%s'
% (' (skipping adult)' if partial_view else ''),
video_id, headers={
'Referer': url,
}, query=query)
if xpath_element(video_xml, './track/video/file') is None:
raise ExtractorError('Specified AfreecaTV video does not exist',
expected=True)
flag = xpath_text(video_xml, './track/flag', 'flag', default=None)
if flag and flag == 'SUCCEED':
break
if flag == 'PARTIAL_ADULT':
self._downloader.report_warning(
'In accordance with local laws and regulations, underage users are restricted from watching adult content. '
'Only content suitable for all ages will be downloaded. '
'Provide account credentials if you wish to download restricted content.')
partial_view = True
continue
elif flag == 'ADULT':
error = 'Only users older than 19 are able to watch this video. Provide account credentials to download this content.'
else:
error = flag
raise ExtractorError(
'%s said: %s' % (self.IE_NAME, error), expected=True)
else:
raise ExtractorError('Unable to download video info')
video_element = video_xml.findall(compat_xpath('./track/video'))[-1]
if video_element is None or video_element.text is None:
raise ExtractorError(
'Video %s does not exist' % video_id, expected=True)
video_url = video_element.text.strip()
title = xpath_text(video_xml, './track/title', 'title', fatal=True)
title = xpath_text(video_xml, './track/title', 'title')
uploader = xpath_text(video_xml, './track/nickname', 'uploader')
uploader_id = xpath_text(video_xml, './track/bj_id', 'uploader id')
duration = int_or_none(xpath_text(video_xml, './track/duration',
'duration'))
duration = int_or_none(xpath_text(
video_xml, './track/duration', 'duration'))
thumbnail = xpath_text(video_xml, './track/titleImage', 'thumbnail')
entries = []
for i, video_file in enumerate(video_xml.findall('./track/video/file')):
video_key = self.parse_video_key(video_file.get('key', ''))
if not video_key:
continue
entries.append({
'id': '%s_%s' % (video_id, video_key.get('part', i + 1)),
common_entry = {
'uploader': uploader,
'uploader_id': uploader_id,
'thumbnail': thumbnail,
}
info = common_entry.copy()
info.update({
'id': video_id,
'title': title,
'upload_date': video_key.get('upload_date'),
'duration': int_or_none(video_file.get('duration')),
'url': video_file.text,
'duration': duration,
})
if not video_url:
entries = []
file_elements = video_element.findall(compat_xpath('./file'))
one = len(file_elements) == 1
for file_num, file_element in enumerate(file_elements, start=1):
file_url = url_or_none(file_element.text)
if not file_url:
continue
key = file_element.get('key', '')
upload_date = self._search_regex(
r'^(\d{8})_', key, 'upload date', default=None)
file_duration = int_or_none(file_element.get('duration'))
format_id = key if key else '%s_%s' % (video_id, file_num)
if determine_ext(file_url) == 'm3u8':
formats = self._extract_m3u8_formats(
file_url, video_id, 'mp4', entry_protocol='m3u8_native',
m3u8_id='hls',
note='Downloading part %d m3u8 information' % file_num)
else:
formats = [{
'url': file_url,
'format_id': 'http',
}]
if not formats:
continue
self._sort_formats(formats)
file_info = common_entry.copy()
file_info.update({
'id': format_id,
'title': title if one else '%s (part %d)' % (title, file_num),
'upload_date': upload_date,
'duration': file_duration,
'formats': formats,
})
entries.append(file_info)
entries_info = info.copy()
entries_info.update({
'_type': 'multi_video',
'entries': entries,
})
return entries_info
info = {
'id': video_id,
'title': title,
@ -131,120 +351,17 @@ class AfreecaTVIE(InfoExtractor):
'thumbnail': thumbnail,
}
if len(entries) > 1:
info['_type'] = 'multi_video'
info['entries'] = entries
elif len(entries) == 1:
info['url'] = entries[0]['url']
info['upload_date'] = entries[0].get('upload_date')
if determine_ext(video_url) == 'm3u8':
info['formats'] = self._extract_m3u8_formats(
video_url, video_id, 'mp4', entry_protocol='m3u8_native',
m3u8_id='hls')
else:
raise ExtractorError(
'No files found for the specified AfreecaTV video, either'
' the URL is incorrect or the video has been made private.',
expected=True)
return info
class AfreecaTVGlobalIE(AfreecaTVIE):
IE_NAME = 'afreecatv:global'
_VALID_URL = r'https?://(?:www\.)?afreeca\.tv/(?P<channel_id>\d+)(?:/v/(?P<video_id>\d+))?'
_TESTS = [{
'url': 'http://afreeca.tv/36853014/v/58301',
'info_dict': {
'id': '58301',
'title': 'tryhard top100',
'uploader_id': '36853014',
'uploader': 'makgi Hearthstone Live!',
},
'playlist_count': 3,
}]
def _real_extract(self, url):
channel_id, video_id = re.match(self._VALID_URL, url).groups()
video_type = 'video' if video_id else 'live'
query = {
'pt': 'view',
'bid': channel_id,
}
if video_id:
query['vno'] = video_id
video_data = self._download_json(
'http://api.afreeca.tv/%s/view_%s.php' % (video_type, video_type),
video_id or channel_id, query=query)['channel']
if video_data.get('result') != 1:
raise ExtractorError('%s said: %s' % (self.IE_NAME, video_data['remsg']))
title = video_data['title']
info = {
'thumbnail': video_data.get('thumb'),
'view_count': int_or_none(video_data.get('vcnt')),
'age_limit': int_or_none(video_data.get('grade')),
'uploader_id': channel_id,
'uploader': video_data.get('cname'),
}
if video_id:
entries = []
for i, f in enumerate(video_data.get('flist', [])):
video_key = self.parse_video_key(f.get('key', ''))
f_url = f.get('file')
if not video_key or not f_url:
continue
entries.append({
'id': '%s_%s' % (video_id, video_key.get('part', i + 1)),
'title': title,
'upload_date': video_key.get('upload_date'),
'duration': int_or_none(f.get('length')),
'url': f_url,
'protocol': 'm3u8_native',
'ext': 'mp4',
})
app, playpath = video_url.split('mp4:')
info.update({
'id': video_id,
'title': title,
'duration': int_or_none(video_data.get('length')),
})
if len(entries) > 1:
info['_type'] = 'multi_video'
info['entries'] = entries
elif len(entries) == 1:
i = entries[0].copy()
i.update(info)
info = i
else:
formats = []
for s in video_data.get('strm', []):
s_url = s.get('purl')
if not s_url:
continue
stype = s.get('stype')
if stype == 'HLS':
formats.extend(self._extract_m3u8_formats(
s_url, channel_id, 'mp4', m3u8_id=stype, fatal=False))
elif stype == 'RTMP':
format_id = [stype]
label = s.get('label')
if label:
format_id.append(label)
formats.append({
'format_id': '-'.join(format_id),
'url': s_url,
'tbr': int_or_none(s.get('bps')),
'height': int_or_none(s.get('brt')),
'url': app,
'ext': 'flv',
'rtmp_live': True,
})
self._sort_formats(formats)
info.update({
'id': channel_id,
'title': self._live_title(title),
'is_live': True,
'formats': formats,
'play_path': 'mp4:' + playpath,
'rtmp_live': True, # downloading won't end without this
})
return info

View File

@ -15,12 +15,12 @@ class AirMozillaIE(InfoExtractor):
_VALID_URL = r'https?://air\.mozilla\.org/(?P<id>[0-9a-z-]+)/?'
_TEST = {
'url': 'https://air.mozilla.org/privacy-lab-a-meetup-for-privacy-minded-people-in-san-francisco/',
'md5': '2e3e7486ba5d180e829d453875b9b8bf',
'md5': '8d02f53ee39cf006009180e21df1f3ba',
'info_dict': {
'id': '6x4q2w',
'ext': 'mp4',
'title': 'Privacy Lab - a meetup for privacy minded people in San Francisco',
'thumbnail': r're:https?://vid\.ly/(?P<id>[0-9a-z-]+)/poster',
'thumbnail': r're:https?://.*/poster\.jpg',
'description': 'Brings together privacy professionals and others interested in privacy at for-profits, non-profits, and NGOs in an effort to contribute to the state of the ecosystem...',
'timestamp': 1422487800,
'upload_date': '20150128',
@ -34,21 +34,13 @@ class AirMozillaIE(InfoExtractor):
def _real_extract(self, url):
display_id = self._match_id(url)
webpage = self._download_webpage(url, display_id)
video_id = self._html_search_regex(r'//vid.ly/(.*?)/embed', webpage, 'id')
video_id = self._html_search_regex(r'//vid\.ly/(.*?)/embed', webpage, 'id')
embed_script = self._download_webpage('https://vid.ly/{0}/embed'.format(video_id), video_id)
jwconfig = self._search_regex(r'\svar jwconfig = (\{.*?\});\s', embed_script, 'metadata')
metadata = self._parse_json(jwconfig, video_id)
formats = [{
'url': source['file'],
'ext': source['type'],
'format_id': self._search_regex(r'&format=(.*)$', source['file'], 'video format'),
'format': source['label'],
'height': int(source['label'].rstrip('p')),
} for source in metadata['playlist'][0]['sources']]
self._sort_formats(formats)
jwconfig = self._parse_json(self._search_regex(
r'initCallback\((.*)\);', embed_script, 'metadata'), video_id)['config']
info_dict = self._parse_jwplayer_data(jwconfig, video_id)
view_count = int_or_none(self._html_search_regex(
r'Views since archived: ([0-9]+)',
webpage, 'view count', fatal=False))
@ -58,17 +50,17 @@ class AirMozillaIE(InfoExtractor):
r'Duration:\s*(\d+\s*hours?\s*\d+\s*minutes?)',
webpage, 'duration', fatal=False))
return {
info_dict.update({
'id': video_id,
'title': self._og_search_title(webpage),
'formats': formats,
'url': self._og_search_url(webpage),
'display_id': display_id,
'thumbnail': metadata['playlist'][0].get('image'),
'description': self._og_search_description(webpage),
'timestamp': timestamp,
'location': self._html_search_regex(r'Location: (.*)', webpage, 'location', default=None),
'duration': duration,
'view_count': view_count,
'categories': re.findall(r'<a href=".*?" class="channel">(.*?)</a>', webpage),
}
})
return info_dict

View File

@ -0,0 +1,53 @@
# coding: utf-8
from __future__ import unicode_literals
from .common import InfoExtractor
from ..compat import compat_str
from ..utils import (
float_or_none,
try_get,
)
class AliExpressLiveIE(InfoExtractor):
_VALID_URL = r'https?://live\.aliexpress\.com/live/(?P<id>\d+)'
_TEST = {
'url': 'https://live.aliexpress.com/live/2800002704436634',
'md5': 'e729e25d47c5e557f2630eaf99b740a5',
'info_dict': {
'id': '2800002704436634',
'ext': 'mp4',
'title': 'CASIMA7.22',
'thumbnail': r're:http://.*\.jpg',
'uploader': 'CASIMA Official Store',
'timestamp': 1500717600,
'upload_date': '20170722',
},
}
def _real_extract(self, url):
video_id = self._match_id(url)
webpage = self._download_webpage(url, video_id)
data = self._parse_json(
self._search_regex(
r'(?s)runParams\s*=\s*({.+?})\s*;?\s*var',
webpage, 'runParams'),
video_id)
title = data['title']
formats = self._extract_m3u8_formats(
data['replyStreamUrl'], video_id, 'mp4',
entry_protocol='m3u8_native', m3u8_id='hls')
return {
'id': video_id,
'title': title,
'thumbnail': data.get('coverUrl'),
'uploader': try_get(
data, lambda x: x['followBar']['name'], compat_str),
'timestamp': float_or_none(data.get('startTimeLong'), scale=1000),
'formats': formats,
}

Some files were not shown because too many files have changed in this diff Show More