Compare commits

..

7651 Commits

Author SHA1 Message Date
a084c80f7b [YouTube] Fix 680069a, excess min_ver
Resolves #33125.
2025-05-04 12:51:54 +01:00
e102b9993a [workflows/ci.yml] Move pinned Ubuntu runner images from withdrawn 20.4 to 22.04
* fix consequent missing `python-is-python2` package
2025-05-03 18:33:39 +01:00
680069a149 [YouTube] Improve n-sig function extraction for player aa3fc80b
Resolves #33123
2025-05-03 00:09:21 +01:00
4a31290ae1 [YouTube] Delete cached problem nsig cache data on descrambling error
* inspired by yt-dlp/yt-dlp#12750
2025-05-03 00:09:21 +01:00
3a42f6ad37 [YouTube] Cache signature timestamp from player JS
* if the YT webpage can't be loaded, getting the `sts` requires loading the
player JS: this caches it
* based on yt-dlp/yt-dlp#13047, thx bashonly
2025-05-03 00:09:21 +01:00
ec75141bf0 [Cache] Add clear function 2025-05-03 00:09:20 +01:00
c052a16f72 [JSInterp] Add tests and relevant functionality from yt-dlp
* thx seproDev, bashonly: yt-dlp/yt-dlp#12760, yt-dlp/yt-dlp#12761:
  - Improve nested attribute support
  - Pass global stack when extracting objects
  - interpret_statement: Match attribute before indexing
  - Fix assignment to array elements with nested brackets
  - Add new signature tests
  - Invalidate JS function cache
  - Avoid testdata dupes now that we cache by URL

* rework nsig function name search
* fully fixes #33102
* update cache required versions
* update program version
2025-04-08 01:59:00 +01:00
bd2ded59f2 [JSInterp] Improve unary operators; add ! 2025-04-08 01:59:00 +01:00
16b7e97afa [JSInterp] Add _separate_at_op() 2025-04-08 01:59:00 +01:00
d21717978c [JSInterp] Improve JS classes, etc 2025-04-08 01:59:00 +01:00
7513413794 [JSInterp] Reorganise some declarations to align better with yt-dlp 2025-04-08 01:59:00 +01:00
67dbfa65f2 [InfoExtractor] Fix merging subtitles to empty target 2025-04-08 01:59:00 +01:00
6eb6d6dff5 [InfoExtractor] Use local variants for remaining parent method calls
* ... where defined
2025-04-08 01:59:00 +01:00
6c40d9f847 [YouTube] Remove remaining hard-coded API keys
* no longer required for these cases
2025-04-08 01:59:00 +01:00
1b08d3281d [YouTube] Fix playlist continuation extraction
* thx coletdjnz, bashonly: yt-dlp/yt-dlp#12777
2025-04-08 01:59:00 +01:00
32b8d31780 [YouTube] Support shorts playlist
* only 1..100: yt-dlp/yt-dlp#11130
2025-04-08 01:59:00 +01:00
570b868078 [cache] Use esc_rfc3986 to encode cache key 2025-04-08 01:59:00 +01:00
2190e89260 [utils] Support optional safe argument for escape_rfc3986() 2025-04-08 01:59:00 +01:00
7e136639db [compat] Improve Py2 compatibility for URL Quoting 2025-04-08 01:59:00 +01:00
cedeeed56f [cache] Align further with yt-dlp
* use compat_os_makedirs
* support non-ASCII characters in cache key
* improve logging
2025-04-08 01:59:00 +01:00
add4622870 [compat] Add compat_os_makedirs
* support exists_ok parameter in Py < 3.2
2025-04-08 01:59:00 +01:00
9a6ddece4d [core] Refactor message routines to align better with yt-dlp
* in particular, support `only_once` in the same methods
2025-04-08 01:59:00 +01:00
3eb8d22ddb [JSInterp] Temporary fix for #33102 2025-03-31 04:21:09 +01:00
4e714f9df1 [Misc] Correct [_]IE_DESC/NAME in a few IEs
* thx seproDev, yt-dlp/yt-dlp/pull/12694/commits/ae69e3c
* also add documenting comment in `InfoExtractor`
2025-03-26 12:47:19 +00:00
c1ea7f5a24 [ITV] Mark ITVX not working
* update old shim
* correct [_]IE_DESC
2025-03-26 12:17:49 +00:00
2b4fbfce25 [YouTube] Support player 4fcd6e4a
thx seproDev, bashonly: yt-dlp/yt-dlp#12748
2025-03-26 02:27:25 +00:00
1bc45b8b6c [JSInterp] Use , for join() with null/undefined argument
Eg: [1,2,3].join(null) -> '1,2,3'
2025-03-25 22:35:06 +00:00
b982d77d0b [YouTube] Align signature tests with yt-dlp
thx bashonly, yt-dlp/yt-dlp#12725
2025-03-25 22:35:06 +00:00
c55dbf4838 [YouTube] Update signature extraction for players 643afba4, 363db69b 2025-03-25 22:35:06 +00:00
087d865230 [YouTube] Support new player URL patterns 2025-03-25 22:35:06 +00:00
a4fc1151f1 [JSInterp] Improve indexing
* catch invalid list index with `ValueError` (eg [1, 2]['ab'] -> undefined)
* allow assignment outside existing list (eg var l = [1,2]; l[9] = 0;)
2025-03-25 22:35:05 +00:00
a464c159e6 [YouTube] Make _extract_player_info() use _search_regex() 2025-03-25 22:35:05 +00:00
7dca08eff0 [YouTube] Also get original of translated automatic captions 2025-03-25 22:35:05 +00:00
2239ee7965 [YouTube] Get subtitles/automatic captions from both web and API responses 2025-03-25 22:35:05 +00:00
da7223d4aa [YouTube] Improve support for tce-style player JS
* improve extraction of global "useful data" Array from player JS
* also handle tv-player and add tests: thx seproDev (yt-dlp/yt-dlp#12684)

Co-Authored-By: sepro <sepro@sepr0.com>
2025-03-21 16:26:25 +00:00
37c2440d6a [YouTube] Update player client data
thx seproDev (yt-dlp/yt-dlp#12603)

Co-authored-by: sepro <sepro@sepr0.com>
2025-03-21 16:13:24 +00:00
420d53387c [JSInterp] Improve tests
* from yt-dlp/yt-dlp#12313
* also fix d7c2708
2025-03-11 02:00:24 +00:00
32f89de92b [YouTube] Update TVHTML5 client parameters
* resolves #33078
2025-03-11 02:00:24 +00:00
283dca56fe [YouTube] Initially support tce-style player JS
* resolves #33079
2025-03-11 02:00:24 +00:00
422b1b31cf [YouTube] Temporarily redirect from tce-style player JS 2025-03-11 02:00:24 +00:00
1dc27e1c3b [JSInterp] Make indexing error handling more conformant
* by default TypeError -> undefined, else raise
* set allow_undefined=True/False to override
2025-03-11 02:00:24 +00:00
af049e309b [JSInterp] Handle undefined, etc, passed to JS_RegExp and Exception 2025-03-11 02:00:24 +00:00
94849bc997 [JSInterp] Improve Date processing
* add JS_Date class implementing JS Date
* support constructor args other than date string
* support static methods of Date
* Date objects are still automatically coerced to timestamp before using in JS.
2025-03-11 02:00:24 +00:00
974c7d7f34 [compat] Fix inheriting from compat_collections_chain_map
* see ytdl-org/youtube-dl#33079#issuecomment-2704038049
2025-03-11 02:00:24 +00:00
8738407d77 [compat] Support zstd Content-Encoding
* see RFC 8878 7.2
2025-03-11 02:00:24 +00:00
cecaa18b80 [compat] Clean-up
* make workaround_optparse_bug9161 private
* add comments
* avoid leaving test objects behind
2025-03-11 02:00:24 +00:00
673277e510 [YouTube] Fix 91b1569 2025-02-28 01:02:20 +00:00
91b1569f68 [YouTube] Fix channel playlist extraction (#33074)
* [YouTube] Extract playlist items from LOCKUP_VIEW_MODEL_...
* resolves #33073
* thx seproDev (yt-dlp/yt-dlp#11615)

Co-authored-by: sepro <sepro@sepr0.com>
2025-02-28 00:02:10 +00:00
711e72c292 [JSInterp] Fix bit-shift coercion for player 9c6dfc4a 2025-02-07 00:01:50 +00:00
26b6f15d14 [compat] Make casefold private
* if required, not supported:
`from youtube_dl.casefold import _casefold as casefold`
2025-02-07 00:01:50 +00:00
5975d7bb96 [YouTube] Use X-Goog-Visitor-Id
* required with tv player client
* resolves #33030
2025-01-22 06:52:40 +00:00
63fb0fc415 [YouTube] Retain .videoDetails members from all player responses 2025-01-20 13:23:54 +00:00
b09442a2f4 [YouTube] Also use ios client when is_live 2025-01-19 01:18:34 +00:00
55ad8a24ca [YouTube] Support ... /feeds/videos.xml?playlist_id={pl_id} 2025-01-15 03:22:56 +00:00
21fff05121 [YouTube] Switch to TV API client
* thx yt-dlp/yt-dlp#12059
2025-01-15 03:19:15 +00:00
1036478d13 [YouTube] Endure subtitle URLs are complete
* WEB URLs are, MWEB not
* resolves #33017
2025-01-06 01:39:04 +00:00
00ad2b8ca1 [YouTube] Refactor subtitle processing
* move to internal function
* use `traverse-obj()`
2025-01-06 01:24:30 +00:00
ab7c61ca29 [YouTube] Apply code style changes, trailing commas, etc 2025-01-06 01:22:16 +00:00
176fc2cb00 [YouTube] Avoid early crash if webpage can't be read
* see issue #33013
2024-12-31 14:51:29 +00:00
d55d1f423d [YouTube] Always extract using MWEB API client
* temporary fix-up for 403 on download
* MWEB parameters from yt-dlp 2024-12-06
2024-12-16 12:38:51 +00:00
eeafbbc3e5 [YouTube] Fix signature function extraction for 2f1832d2
* `_` was omitted from patterns
* thx yt-dlp/yt-dlp#11801

Co-authored-by: bashonly
2024-12-16 12:38:51 +00:00
cd7c7b5edb [YouTube] Simplify pattern for nsig function name extraction 2024-12-16 12:38:51 +00:00
eed784e15f [YouTube] Pass nsig value as return hook, fixes player 3bb1f723 2024-12-16 12:38:51 +00:00
b4469a0f65 [YouTube] Handle player 3bb1f723
* fix signature code extraction
* raise if n function returns input value
* add new tests from yt-dlp

Co-authored-by: bashonly
2024-12-16 12:38:51 +00:00
ce1e556b8f [jsinterp] Add return hook for player 3bb1f723
* set var `_ytdl_do_not_return` to a specific value in the scope of a function
* if an expression to be returned has that value, `return` becomes `void`
2024-12-16 12:38:51 +00:00
f487b4a02a [jsinterp] Strip /* comments */ when parsing
* NB: _separate() is looking creaky
2024-12-16 12:38:51 +00:00
60835ca16c [jsinterp] Fix and improve "methods"
* push, unshift return new length
* impove edge cases for push/pop, shift/unshift, forEach, indexOf, charCodeAt
* increase test coverage
2024-12-16 12:38:51 +00:00
94fd774608 [jsinterp] Fix and improve split/join
* improve split/join edge cases
* correctly implement regex split (not like re.split)
2024-12-16 12:38:51 +00:00
5dee6213ed [jsinterp] Fix and improve arithmetic operations
* addition becomes concat with a string operand
* improve handling of edgier cases
* arithmetic in float like JS (more places need cast to int?)
* increase test coverage
2024-12-16 12:38:51 +00:00
81e64cacf2 [jsinterp] Support multiple indexing (eg a[1][2])
* extend single indexing with improved RE (should probably use/have used _separate_at_paren())
* fix some cases that should have given undefined, not throwing
* standardise RE group names
* support length of objects, like {1: 2, 3: 4, length: 42}
2024-12-16 12:38:51 +00:00
c1a03b1ac3 [jsinterp] Fix and improve loose and strict equality operations
* reimplement loose equality according to MDN (eg, 1 == "1")
* improve strict equality (eg, "abc" === "abc" but 'abc' is not 'abc')
* add tests for above
2024-12-16 12:38:51 +00:00
118c6d7a17 [jsinterp] Implement typeof operator 2024-12-16 12:38:51 +00:00
f28d7178e4 [InfoExtractor] Use kwarg maxsplit for re.split
* May become kw-only in future Pythons
2024-12-16 12:38:51 +00:00
c5098961b0 [Youtube] Rework n function extraction pattern
Now also succeeds with player b12cc44b
2024-08-06 20:59:09 +01:00
dbc08fba83 [jsinterp] Improve slice implementation for player b12cc44b
Partly taken from yt-dlp/yt-dlp#10664, thx seproDev
        Fixes #32896
2024-08-06 20:51:38 +01:00
71223bff39 [Youtube] Fix nsig extraction for player 20dfca59 (#32891)
* dirkf's patch for nsig extraction
* add generic search per  yt-dlp/yt-dlp/pull/10611 - thx bashonly

---------

Co-authored-by: dirkf <fieldhouse@gmx.net>
2024-08-01 19:18:34 +01:00
e1b3fa242c [Youtube] Find n function name in player 3400486c
Fixes #32877
2024-07-25 00:16:00 +01:00
451046d62a [Youtube] Make n-sig throttling diagnostic up-to-date 2024-07-24 14:33:34 +01:00
16f5bbc464 [YouTube] Fix nsig processing for player b22ef6e7
* improve extraction of function name (like yt-dlp/yt-dlp#10390)
* always use JSInterp to extract function code (yt-dlp/yt-dlp#10396, thx seproDev, pukkandan)
2024-07-11 00:50:46 +01:00
d35ce6ce95 [jsinterp] Support functionality for player b22ef6e7
* support `prototype` for call() and apply() (yt-dlp/yt-dlp#10392, thx Grub4k)
* map JS `Array` to `list`
2024-07-11 00:50:46 +01:00
76ac69917e [jsinterp] Further improve expression parsing (fix fd8242e)
Passes tests from yt-dlp
2024-07-11 00:50:46 +01:00
756f6b45c7 [jsinterp] Re-align JSInterp and tests (esp.) with yt-dlp
Thx: various yt-dlp authors
2024-07-11 00:50:46 +01:00
43a74c5fa5 [core] Address gaps in allowed extensions
Adds some extensions missing in 4652109643
(from yt-dlp/yt-dlp#10362)

Authored by: bashonly
Co-authored by: dirkf
2024-07-11 00:50:46 +01:00
a452f9437c [core] Fix PR #32830 for fixed extensionless output template 2024-07-07 22:33:32 +01:00
36801c62df [YandexMusic] Save track version in the title field
PR #32837
* Add track version to track title
2024-07-07 20:18:33 +01:00
f4b47754d9 [YandexMusic] Download music in High Quality (320 Kbit/s)
PR #31159
2024-07-06 11:04:36 +01:00
37cea84f77 [core,utils] Support unpublicised --no-check-extensions 2024-07-02 15:38:50 +01:00
4652109643 [core,utils] Implement unsafe file extension mitigation
* from https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-79w7-vh3h-8g4, thx grub4k
2024-07-02 15:38:50 +01:00
3c466186a8 [utils] Back-port Namespace and MEDIA_EXTENSIONS from yt-dlp
Thx pukkandan
* Namespace: https://github.com/yt-dlp/yt-dlp/commit/591bb9d355
* MEDIA_EXTENSIONS: https://github.com/yt-dlp/yt-dlp/commit/8dc5930511
2024-07-02 15:38:50 +01:00
4d05f84325 [PalcoMP3] Conform to new linter rule
* no space after @ in decorator
2024-06-20 20:03:49 +01:00
e0094e63c3 [jsinterp] Various tweaks
* treat Infinity like NaN
* cache operator list
2024-06-20 20:03:49 +01:00
fd8242e3ef [jsinterp] Fix and improve expression parsing
* improve BODMAS (fixes https://github.com/ytdl-org/youtube-dl/issues/32815)
* support more weird expressions with multiple unary ops
2024-06-20 20:03:49 +01:00
ad01fa6cca [jsinterp] Add Debugger from yt-dlp
* https://github.com/yt-dlp/yt-dlp/commit/8f53dc4
* thx pukkandan
2024-06-20 20:03:49 +01:00
2eac0fa379 [utils] Save orig_msg in ExtractorError 2024-06-20 20:03:49 +01:00
0153b387e5 [VidLii] Add 720p support (#30924)
* [VidLii] Add HD support  (yt-dlp backport-ish)

* Also fix a bug with the view count

---------

Co-authored-by: dirkf <fieldhouse@gmx.net>
2024-06-11 13:21:39 +01:00
a48fe7491d [ORF] Skip tests with limited availability 2024-06-11 12:52:13 +01:00
e20ca543f0 [ORF] Re-factor and updateORFFM4StoryIE
* fix getting media via DASH instead of inaccessible mp4
* also get in-page YT media
2024-06-11 12:52:13 +01:00
e39466051f [ORF] Support sound.orf.at, updating ORFRadioIE
* maintain support for xx.orf.at/player/... URLs
* add `ORFRadioCollectionIE` to support playlists in ORF Sound
* back-port and re-work `ORFPodcastIE` from https://github.com/yt-dlp/yt-dlp/pull/8486, thx Esokrates
2024-06-11 12:52:13 +01:00
d95c0d203f [ORF] Support on.orf.at, replacing ORFTVthekIE
* add `ORFONIE`, back-porting yt-dlp PR https://github.com/yt-dlp/yt-dlp/pull/9113 and friends: thx HobbyistDev, TuxCoder, seproDev
* re-factor to support livestreams via new `ORFONliveIE`
2024-06-11 12:52:13 +01:00
3bde6a5752 [test] Improve download test
* skip reason can't be unicode in Py2
* remove duplicate assert...Equal functions
2024-06-11 12:52:13 +01:00
50f6c5668a [core] Re-factor with _fill_common_fields() as used in yt-dlp 2024-06-11 12:52:13 +01:00
b4ff08bd2d [core] Safer handling of nested playlist data 2024-06-11 12:52:13 +01:00
88bd8b9f87 [mixcloud] updated mixcloud API server address (#32557)
* updated mixcloud API server address
* fix tests
* etc

---------

Co-authored-by: dirkf <fieldhouse@gmx.net>
2024-06-11 12:38:24 +01:00
21924742f7 [InfoExtractor] Misc yt-dlp back-ports, etc
* add _yes_playlist() method
* avoid crash using _NETRC_MACHINE
* use _search_json() in _search_nextjs_data()
* _search_nextjs_data() default is JSON, not text
* test for above
2024-05-30 15:46:36 +01:00
768ccccd9b [compat] Avoid type comparison in compat_ord
NB This isn't actually a compat fn; it should be utils.int_from_int_or_char
2024-05-30 15:46:36 +01:00
eee9a247eb [utils] Split out traversal.py dummy and traversal tests 2024-05-30 15:46:36 +01:00
34484e49f5 [compat] Improve compat_etree_iterfind for Py2.6
Adapted from https://raw.githubusercontent.com/python/cpython/2.7/Lib/xml/etree/ElementPath.py
2024-05-30 15:46:36 +01:00
06da64ee51 [utils] Update traverse_obj() from yt-dlp
* remove `is_user_input` option per https://github.com/yt-dlp/yt-dlp/pull/8673
* support traversal of compat_xml_etree_ElementTree_Element per https://github.com/yt-dlp/yt-dlp/pull/8911
* allow un/branching using all and any per https://github.com/yt-dlp/yt-dlp/pull/9571
* support traversal of compat_cookies.Morsel and multiple types in `set()` keys per https://github.com/yt-dlp/yt-dlp/pull/9577
thx Grub4k for these
* also, move traversal tests to a separate class
* allow for unordered dicts in tests for Py<3.7
2024-05-30 15:46:36 +01:00
a08f2b7e45 [workflows/ci.yml] Temporary workaround for Python 3.5 _pip_ failures
https://github.com/actions/setup-python/issues/866
2024-05-15 16:57:59 +01:00
668332b973 [YouPorn] Add playlist extractors
* YouPornCategoryIE
* YouPornChannelIE
* YouPornCollectionIE
* YouPornStarIE
* YouPornTagIE
* YouPornVideosIE,
2024-04-22 01:34:26 +01:00
0b2ce3685e [YouPorn] Improve extraction
* detect unwatchable videos
* improve duration extraction
* fix count extraction and support large values
* detect and remove SEO spam boilerplate description
2024-04-22 01:34:26 +01:00
c2766cb80e [test/test_download] Support 'playlist_maxcount:count' expected value
* parallel to `playlist_mincount'
* specify both for a range of playlist lengths
* if max < min the test will always fail!
2024-04-22 01:34:26 +01:00
eb38665438 [YouPorn] Incorporate yt-dlp PR 8827
* from https://github.com/yt-dlp/yt-dlp/pull/8827
* extract from webpage instead of broken API URL
* thx The-MAGI
2024-04-22 01:34:26 +01:00
e0727e4ab6 [postprocessor/ffmpeg] Fix finding ffprobe (bug in 21792b8)
Fixes 21792b88b7 (commitcomment-140705274), thx: vonProteus
2024-04-07 15:33:30 +01:00
4ea59c6107 [utils] Fix crash in _report_ignoring_subs from c58b655 (#32762)
Align `utils.bug_reports_message()` with yt-dlp https://github.com/yt-dlp/yt-dlp/commit/5873d4ccdd, thanks fstirlitz

---------

Co-authored-by: dirkf <fieldhouse@gmx.net>
2024-04-05 15:25:29 +01:00
21792b88b7 [external/FFmpeg] Fix and improve --ffmpeg-location handling
* pass YoutubeDL (FileDownloader) to FFmpegPostProcessor constructor
* consolidate path search in FFmpegPostProcessor
* make availability of FFmpegFD depend on existence of FFmpegPostProcessor
* detect ffmpeg executable on instantiation of FFmpegFD
* resolves #32735
2024-03-27 13:11:17 +00:00
d8f134a664 [downloader/external] Fix "Resource Warning" in downloader test
* add compat_subprocess_Popen context manager
* apply context manager in FFmpegFD._call_downloader()
2024-03-27 13:11:17 +00:00
31a15a7c8d [compat] Simplify/fix compat_html_parser_HTMLParseError 2024-03-27 13:11:17 +00:00
19dc10b986 [utils] Apply compat_contextlib_suppress 2024-03-27 13:11:17 +00:00
182f63e82a [compat] Add compat_contextlib_suppress
with compat_contextlib_suppress(*Exceptions):
    # code that fails silently for any of Exceptions
2024-03-27 13:11:17 +00:00
71211e7db7 [Youtube] Fix unwanted private method __ie_msg in f8b0135850
Fixes `AttributeError no attribute '_YoutubeIE__ie_msg'` if unable to decode n-parameter
2024-03-23 15:30:13 +00:00
a96a45b2cd [Vimeo] Improve config extraction (#32742)
* update for more robust json parsing
2024-03-12 11:44:13 +00:00
820fae3b3a [Videa] Fix extraction
* update API URL
* from https://github.com/yt-dlp/yt-dlp/pull/8003
* thanks to the authors!

Closes yt-dlp/7427
Authored by: hatsomatt, aky-01
2024-03-08 13:14:52 +00:00
aef24d97e9 [Videa] Align with yt-dlp 2024-03-08 13:14:52 +00:00
f7b30e3f73 [XFileShare] Update extractor for 2024
* simplify aa_decode()
* review and update supported sites and tests
* in above, include FileMoon.sx, and remove separate module
* incorporate changes from yt-dlp
* allow for decoding multiple scripts (eg, FileMoon)
* use new JWPlayer extraction
2024-03-08 13:03:42 +00:00
f66372403f [InfoExtractor] Rework and improve JWPlayer extraction
* use traverse_obj() and _search_json()
* support playlist `.load({**video1},{**video2}, ...)`
* support transform_source=... for _extract_jwplayer_data()
2024-03-08 13:03:42 +00:00
7216fa2ac4 [InfoExtractor] Add _search_json()
* uses the error diagnostic to truncate the JSON string
* may be confused by non-C-Pythons
2024-03-08 13:03:42 +00:00
acc383b9e3 [utils] Let int_or_none() accept a base, like int() 2024-03-08 13:03:42 +00:00
f0812d7848 [utils] Handle user:pass in URLs (#28801)
* Handle user:pass in URLs

Fixes "nonnumeric port" errors when youtube-dl is given URLs with
usernames and passwords such as:

    http://username:password@example.com/myvideo.mp4

Refs:
- https://en.wikipedia.org/wiki/Basic_access_authentication
- https://tools.ietf.org/html/rfc1738#section-3.1
- https://docs.python.org/3.8/library/urllib.parse.html#urllib.parse.urlsplit

Fixes #18276 (point 4)
Fixes #20258
Fixes #26211 (see comment)

* Align code with yt-dlp

---------

Co-authored-by: dirkf <fieldhouse@gmx.net>
2024-03-04 01:27:55 +00:00
40bd5c1815 [caffeine.tv] Add new extractor (#32514)
* Add CaffeineTVIE info extractor to support site caffeine.tv

---------

Co-authored-by: dirkf <fieldhouse@gmx.net>
2024-02-22 12:54:07 +00:00
70f230f9cf [GBNews]Add new extractor for GB News TV channel (#29432)
* Add extractor for GB News TV channel

* Support more GBNews URL formats
Allow alphanumeric and _ in place of `shows`, which redirect to site's preferred URL

* Update for 2024
2024-02-22 12:44:00 +00:00
48ddab1f3a [downloader/external] Fix WgetFD proxy (rev 2)
From PR (defunct source), closes #29343.
Matches https://github.com/yt-dlp/yt-dlp/pull/3152
Thx former user kikuyan.
2024-02-21 16:29:08 +00:00
7687389f08 [Vbox7] Improve extraction, adding features from yt-dlp PR #9100
* changes from https://github.com/yt-dlp/yt-dlp/pull/9100 (thx
seproDev):
  - attempt HLS extraction
  - re-enable XFF
  - test `view_count`, `duration` extraction
* improve commenting, error checks
2024-02-19 00:53:22 +00:00
4416f82c80 [Vbox7IE] Sanitise ld+json containing unexpected characters
* based on PR #29680
* added hack to force invoking `transform_source`
* fixes #26218
2024-02-02 12:36:05 +00:00
bdda6b81df [Vbox7IE] Improve extraction
* DASH extraction no longer fails with new range support
* but always find combined formats if available
* suppress ineffective XFF geo-bypass (causes time-outs)
* adapted from https://github.com/ytdl-org/youtube-dl/pull/29680
* thx former GH user kikuyan
2024-02-02 12:36:05 +00:00
1fd8f802b8 [InfoExtractor] Correctly resolve BaseURL in DASH manifest
Specs:
* ISO/IEC 23009-1:2012 section 5.6
* RFC 3986 section 5.
2024-02-02 12:36:05 +00:00
4eaeb9b2c6 [InfoExtractor] Support byte range for DASH
* adapted from https://github.com/ytdl-org/youtube-dl/pull/30279
* thx former GH user kikuyan
2024-02-02 12:36:05 +00:00
bec9180e89 [downloader/dash] Support range in fragment (format f'{start}-{end}')
* adapted from https://github.com/ytdl-org/youtube-dl/pull/30279
 * thx former GH user kikuyan
2024-02-02 12:36:05 +00:00
c58b655a9e [InfoExtractor] Support DASH subtitle extraction (yt-dlp back-port) 2024-02-02 12:36:05 +00:00
dc512e3a8a [YouTube] Fix like_count extraction using likeButtonViewModel
* also fix various tests
* TODO: check against yt-dlp tests
2024-01-22 11:10:34 +00:00
f8b0135850 [YouTube] Rework n-sig processing, realigning with yt-dlp
* apply n-sig before chunked fragments, fixes #32692
2024-01-22 11:10:34 +00:00
640d39f03a [InfoExtractor] Support some warning and ._downloader shortcut methods from yt-dlp 2024-01-22 11:10:34 +00:00
6651871416 [compat] Rework compat for method parameter of compat_urllib_request.Request constructor
* fixes #32573
* does not break `utils.HEADrequest` (eg)
2024-01-22 11:10:34 +00:00
be008e657d [core] Fix format string injection for metadata JSON filename message. 2023-12-06 02:45:41 +00:00
b1bbc1e502 [Epidemic Sound] Add new extractor (#32628)
* Add simple extractor
* Support separate tracks
* Use index as id instead of slug

---------

Co-authored-by: dirkf <fieldhouse@gmx.net>
2023-12-06 01:17:57 +00:00
55a442adae [Imgur] Overhaul extractor module (#32612)
Revise extractors for new API and page formats
2023-12-05 20:02:30 +00:00
c62936a5f2 [telewebion] Fix extraction (#32634)
* [telewebion] fix extraction

Resolves https://github.com/ytdl-org/youtube-dl/issues/5135#issuecomment-932952119

---------

Co-authored-by: dirkf <fieldhouse@gmx.net>
2023-12-02 15:25:09 +00:00
427472351c [utils] Make restricted filenames ignore characters in Unicode categories Mark, Other
Resolves #32629
2023-11-29 22:08:01 +00:00
c6538ed323 [workflows/ci.yml] Use setup-python for now released Python 3.12 2023-11-29 22:08:01 +00:00
8d227cb97b [workflows/ci.yml] Actually use default values for push and pull_request 2023-11-29 22:08:01 +00:00
4e115e18cb [workflows/ci.yml] Run apt-get update before installing 2023-11-29 22:08:01 +00:00
b7fca0fab3 [Youtube] Update consent cookie handling to match site
Apologies for force push!
[skip ci]
2023-11-29 21:43:02 +00:00
00ef748cc0 [downloader] Fix baa6c5e: show ETA of http download as ETA instead of total d/l time 2023-09-24 22:07:47 +01:00
66ab0814c4 [utils] Revert bbd3e7e, updating docstring, test instead 2023-09-03 23:15:19 +01:00
bbd3e7e999 [utils] Properly handle list values in update_url()
An actual list value in a query update could have been treated
as a list of values because of the key:list parse_qs format.
2023-09-03 01:18:22 +01:00
21caaf2380 [test] Remove redundancy from lambda expected value regex 2023-09-03 01:13:40 +01:00
31f50c8194 [S4C] Add thumbnail extraction, extract series as playlist
Based on https://github.com/yt-dlp/yt-dlp/pull/7776: thx ifan-t, bashonly
2023-08-31 23:16:50 +01:00
7d58f0769a [ci.yml] Improve conditions for nosetest installations 2023-08-31 17:16:47 +01:00
86e3cf5e58 [S4C] Add extractor for Sianel Pedwar Cymru
* from https://github.com/yt-dlp/yt-dlp/pull/7730, thx ifan-t, bashonly
2023-08-04 22:54:12 +01:00
2efc8de4d2 [utils] Advertise optional supported Content-Encodings 2023-08-01 01:05:09 +01:00
e4178b5af3 [utils] Add and use filter_dict() from yt-dlp 2023-08-01 01:05:09 +01:00
2d2a4bc832 [utils] Revise isinstance() tests (especially for str/unicode/bytes) to complete Linter fix 2023-08-01 01:05:09 +01:00
7d965e6b65 [utils] Avoid comparing type(var), etc, to pass new Linter rules 2023-08-01 01:05:09 +01:00
abef53466d [utils] Rework URL path munging for ., .. components
* move processing to YoutubeDLHandler
* also process `Location` header for redirect
* use tests from https://github.com/yt-dlp/yt-dlp/pull/7662
2023-07-29 14:27:26 +01:00
e7926ae9f4 [utils] Rework decoding of Content-Encodings
* support nested encodings
* support optional `br` encoding, if brotli package is installed
* support optional 'compress' encoding, if ncompress package is installed
* response `Content-Encoding` has only unprocessed encodings, or removed
* response `Content-Length` is decoded length (usable for filesize metadata)
* use zlib for both deflate and gzip decompression
* some elements taken from yt-dlp: thx especially coletdjnz
2023-07-29 14:27:26 +01:00
87e578c9b8 [workflows/ci.yml] Update to setup-java@v3
* avoid Node 12 deprecation
2023-07-29 14:27:26 +01:00
0861812d72 [build] Fix typo in devscripts/fish-completion.py (fix 2285605) 2023-07-25 15:11:15 +01:00
b870181229 [build] Extend use of devscripts/utils 2023-07-25 13:19:43 +01:00
a25e9f3c84 [compat] Use compat_open() 2023-07-25 13:19:43 +01:00
aac33155e4 [build] Add and use devscripts/utils 2023-07-25 13:19:43 +01:00
2b7dd3b2a2 [utils] Fix update_Request() with empty data (not None) 2023-07-25 13:19:43 +01:00
44faa71b19 [test/test_execution.py] Use compat_subprocess_get_DEVNULL() 2023-07-25 13:19:43 +01:00
7bce2ad441 [build] Fix various Jython CI and test issues 2023-07-25 13:19:43 +01:00
ca71e56c48 [workflows/ci.yml] Build 3.12 with pyenv 2023-07-25 13:19:43 +01:00
2a4e9faa77 [doc] Update developer guidance
* mention pynose
* mention traverse_obj and add/revise examples

[skip ci]
2023-07-25 13:19:43 +01:00
74eef6bb5e [workflows/ci.yml] Extend Python versions
* add 3.10 - 3.12
* use https://pypi.org/project/pynose/ for Py >= 3.9
* test Windows with 3.4
* set defaults (main, both) except push: (all, core)
2023-07-25 13:19:43 +01:00
1fa8b86f0b [utils] Remove stray undocumented Host header in redirect (fix 46fde7c) 2023-07-20 05:29:59 +01:00
b2ba24bb02 [InfoExtractor] Add _match_valid_url() class method and refactor
* API compatible with yt-dlp
* also support Sequence of patterns in _VALID_URL
* one place to compile _VALID_URL
* TODO: remove existing extractor shims
2023-07-19 22:14:50 +01:00
a190b55964 [utils] Fix broken Py 3.11+ compat in traverse_obj()
* inspect.getargspec is missing despite doc claiming backward compat
* replace with emulation of `Signature.bind()`
2023-07-19 22:14:50 +01:00
b2741f2654 [InfoExtractor] Add search methods for Next/Nuxt.js from yt-dlp
* add _search_nextjs_data(), from https://github.com/yt-dlp/yt-dlp/pull/1386
  thanks selfisekai
* add _search_nuxt_data(), from https://github.com/yt-dlp/yt-dlp/pull/1921,
  thanks Lesmiscore, pukkandan
* add tests for the above
* also fix HTML5 type recognition and tests, from
  222a230871,
  thanks Lesmiscore
* update extractors in PR using above, fix tests.
2023-07-19 22:14:50 +01:00
8465222041 [Clipchamp] Add new extractor back-ported from yt-dlp 2023-07-19 22:14:50 +01:00
4339910df3 [DLF] Add site extractors back-ported from yt-dlp
* from https://github.com/yt-dlp/yt-dlp/pull/6697, thanks nick-cd
2023-07-19 22:14:50 +01:00
eaaf4c6736 [Whyp] Add extractor back-ported from yt-dlp
* from https://github.com/yt-dlp/yt-dlp/pull/6803, thanks CoryTibbettsDev
2023-07-19 22:14:50 +01:00
4566e6e53e [GlobalPlayer] Add site extractors back-ported from yt-dlp
* from https://github.com/yt-dlp/yt-dlp/pull/6903, thanks garret1317
2023-07-19 22:14:50 +01:00
1e8ccdd2eb [InfoExtractor] Support groups in _search_regex(), etc 2023-07-19 22:14:50 +01:00
cb9366eda5 [utils] Minor updates (merge_dicts, T)
A couple of mods to ease yt-dlp back-ports:
* add kwargs to merge_dicts:
  `unblank=True` (disallow empty string), `rev=False` (reverse the merge list)
* add `T(x)` shortcut for `{x}`, unsupported in Py2.6
2023-07-19 22:14:50 +01:00
d9d07a9581 [utils] Improve js_to_json, align with yt-dlp
* support variable substitution, from https://github.com/yt-dlp/yt-dlp/pull/#521 etc,
  thanks ChillingPepper, Grub4k, pukkandan
* improve escape handling, from https://github.com/yt-dlp/yt-dlp/pull/#521
  thanks Grub4k
* support template strings from https://github.com/yt-dlp/yt-dlp/pull/6623
  thanks Grub4k
* add limited `!` evaluation (eg, !!0 -> false, see tests)
2023-07-19 22:14:50 +01:00
825a40744b [utils] Align traverse_obj() with yt-dlp
Thanks Grub4k for these:
* traverse `Iterable`s, from https://github.com/yt-dlp/yt-dlp/pull/6902, etc
* traverse `set` key for transformations/filters, `re.Match` group names, from
  776995bc10, etc
* traverse `re.Match`es, from https://github.com/yt-dlp/yt-dlp/pull/5174
* always return list when branching, from https://github.com/yt-dlp/yt-dlp/pull/5170
2023-07-19 22:14:50 +01:00
47214e46d8 [compat] Fix old Pythons broken loading of valueless cookie attributes
Cookie string parsing in Py 2.6.9, probably earlier, requires `=`.
Also 3.2, though the CPython code appears to be OK: 3.1 was also wrong.
2023-07-18 10:50:46 +01:00
1d8d5a93f7 [test] Fixes for old Pythons 2023-07-18 10:50:46 +01:00
1634b1d61e [doc] Warn against setting cookies with --add-header 2023-07-18 10:50:46 +01:00
21438a4194 [downloader/external] Fix cookie support 2023-07-18 10:50:46 +01:00
8334ec961b [core] Process header cookies on loading 2023-07-18 10:50:46 +01:00
3801d36416 [utils] YoutubeDLCookieJar: Add get_cookie_header and get_cookies_for_url methods 2023-07-18 10:50:46 +01:00
b383be9887 [core] Remove Cookie header on redirect to prevent leaks
Adated from yt-dlp/yt-dlp-ghsa-v8mc-9377-rwjj/pull/1/commits/101caac
Thx coletdjnz
2023-07-18 10:50:46 +01:00
46fde7caee [core] Update redirect handling from yt-dlp
* Thx coletdjnz: https://github.com/yt-dlp/yt-dlp/pull/7094
* add test that redirected `POST` loses its `Content-Type`
2023-07-18 10:50:46 +01:00
648dc5304c [compat] Add Request and HTTPClient compat for redirect
* support `method` parameter of `Request.__init__`  (Py 2 and old Py 3)
* support `getcode` method of compat_http_client.HTTPResponse (Py 2)
2023-07-18 10:50:46 +01:00
1720c04dc5 [test] Make skipped tests in test_execution work with Py 2.6 2023-07-18 10:50:46 +01:00
d5ef405c5d [core] Align error reporting methods with yt-dlp 2023-07-18 10:50:46 +01:00
f47fdb9564 [utils] Add {expected_type} and Iterable support to traverse_obj() 2023-07-18 10:50:46 +01:00
b6dff4073d [core] Revert version display from b8a86dc 2023-07-18 10:50:46 +01:00
f24bc9272e [Misc] Fixes for 2.6 compatibility 2023-07-05 22:58:54 +01:00
b08a580906 [workflows/ci.yml] Fix test support for Py 2.6 2023-07-05 22:58:09 +01:00
2500300c2a [workflows/ci.yml] Restore test support for Py 3.2 2023-07-05 22:51:15 +01:00
58fc5bde47 [workflows/ci.yml] Restore test support for Py 3.3, 3.4, and add 2.6 2023-06-23 00:15:06 +01:00
fa7f0effbe [YouTube] Avoid crash in author extraction 2023-06-22 23:14:21 +01:00
ebdc82c586 [workflows/ci.yml] Replace actions/setup-python for legacy Pythons
Thanks MatteoH2O1999: https://github.com/MatteoH2O1999/setup-python
2023-06-22 23:12:22 +01:00
9112e668a5 [YouTube] Improve nsig function name extraction
Fixes player b7910ca8, using `,` vs `;`
See https://github.com/ytdl-org/youtube-dl/issues/32292#issuecomment-1602231170

Co-authored-by: dirkf
2023-06-22 16:46:53 +01:00
07af47960f [YouTube] Improve fix for ae8ba2c
Thx: https://github.com/yt-dlp/yt-dlp/commit/01aba25
2023-06-18 00:52:18 +01:00
ae8ba2c319 [YouTube] Fix KeyError QV in signature extraction failed
* temporarily force missing global definition into sig JS
* improve test: thanks https://github.com/yt-dlp/yt-dlp/issues/7327#issuecomment-1595274615
* resolves #32314
2023-06-17 15:55:19 +01:00
d6433cbb2c [jsinterp] Don't find unrelated objects 2023-06-17 15:46:12 +01:00
ff75c300f5 [jsinterp] Fix test for failed match in extract_object() 2023-06-17 15:34:11 +01:00
a2534f7b88 [jsinterp] Fix div bug breaking player 8c7583ff
Thx bashonly: https://github.com/ytdl-org/youtube-dl/issues/32292#issuecomment-1585639223
Fixes #32292
2023-06-11 17:23:00 +01:00
b8a86dcf1a [core] Revise 1f7c6f8 to help downstream merger (possibly) 2023-05-26 20:25:25 +01:00
2389c7cbd3 [compat] Fix casefold import __all__ syntax in a19855f 2023-05-23 17:11:22 +01:00
ee731f3d00 [ITV] Fix UA capitalisation in 384f632 2023-05-23 16:50:25 +01:00
1f7c6f8b2b [core] Further improve platform debug log
* see d1c6c5c
2023-05-23 16:50:25 +01:00
d89c2137ba [jsinterp] Small updates for a85a875
* update signature tests
* clarify NaN handling
2023-05-23 16:50:25 +01:00
d1c6c5c4d6 [core] Improve platform debug log, based on yt-dlp 2023-05-11 21:17:31 +01:00
6ed3433828 [jsinterp] Add short-cut evaluation for common expression
* special handling for (d%e.length+e.length)%e.length speeds up ~6%
2023-05-11 21:02:01 +01:00
a85a875fef [jsinterp] Handle NaN in bitwise operators
* also add _NaN
* also pull function naming from yt-dlp
2023-05-11 20:59:30 +01:00
11cc3f3ad0 [utils] Fix compiled_regex_type in 249f2b6 2023-05-11 20:53:07 +01:00
64d6dd64c8 [YouTube] Support Releases tab 2023-04-23 22:58:35 +01:00
211cbfd5d4 [jsinterp] Minimally handle arithmetic operator precedence
Resolves #32066
2023-04-21 14:04:30 +01:00
26035bde46 [DashSegmentsFD] Correctly detect errors when fragment_retries == 0
* use the success flag instead of the retry count
* establish the fragment_url outside the retry loop
* only report skipping a fragment once.
* resolves #32033
2023-04-13 00:23:17 +01:00
2da3fa04a6 [YouTube] Simplify signature patterns 2023-04-12 23:53:14 +01:00
735e87adfc [core] Sanitize info dict before dumping JSON (fixes fe7e130) (#32032)
* follow up to fe7e130 which didn't fix everything.

Co-authored-by: dirkf <fieldhouse@gmx.net>
2023-04-12 23:40:38 +01:00
fe7e13066c [core] Add and use sanitize_info() method from yt-dlp 2023-04-10 17:12:31 +01:00
213d1d91bf [core] No longer importing copy 2023-04-06 19:49:46 +01:00
f8253a5289 [core] Avoid deepcopy of ctx dict (fix f35b757) (Pt 2) 2023-04-06 19:42:36 +01:00
d6ae3b77cd [core] Avoid deepcopy of ctx dict (fix f35b757)
* may now contain `LazyList`s
* resolves #31999
2023-04-06 14:56:55 +01:00
9f4d83ff42 [options] Add --mtime option, unsets default --no-mtime
* resolves #1709 (!)
2023-04-05 19:05:16 +01:00
25124bd640 [devscripts] Improve hack to convert command-line options to API options
* define equality for DateRange
* don't show default DateRange
2023-04-05 19:05:16 +01:00
78da22489b [compat] Add and use compat_open() like Py3 open()
* resolves FIXME: ytdl-org/youtube-dl/commit/dfe5fa4
2023-04-05 18:57:37 +01:00
557dbac173 [FragmentFD] Fix iteration with infinite limit
* fixes ytdl-org/youtube-dl/baa6c5e
* resolves #31885
2023-04-05 18:55:41 +01:00
cdf40b6aa6 [test] Update tests for Ubuntu 20.04
* 18.04 test runner was withdrawn
* for now, disable Py 3.3/3.4 tests
2023-04-05 18:54:30 +01:00
3f6d2bd76f [extractor/youtube] Bypass throttling for -f17
and related cleanup

Thanks @AudricV for the finding

Ref: yt-dlp/yt-dlp/commit/c9abebb
2023-03-19 02:29:00 +00:00
88f28f620b [extractor/youtube] Construct fragment list lazily
Ref: yt-dlp/yt-dlp/commit/e389d17
See: yt-dlp/yt-dlp#6517
2023-03-19 02:29:00 +00:00
f35b757c82 [utils] Ensure allow_types for variadic() is a tuple 2023-03-19 02:29:00 +00:00
45495228b7 [downloader/http] Only check for resumability when actually resuming 2023-03-19 02:15:41 +00:00
6fece0a96b [AENetworksBaseIE] Report missing show data instead of crash 2023-03-14 16:23:20 +00:00
70ff013910 [devscripts] Add a hack to convert command-line options to API options 2023-03-14 16:23:20 +00:00
e8de54bce5 [core] Handle /../ sequences in HTTP URLs
* use Python's RFC implementation for embedded sequences
* hack: strip unbalanced leading `../` from path, like eg Firefox

See https://github.com/yt-dlp/yt-dlp/issues/3355
2023-03-14 16:23:20 +00:00
baa6c5e95c [FragmentFD] Respect --no-continue
* discard partial fragment on `--no-continue`
* continue with correct progress display otherwise

Resolves #21467
2023-03-14 16:23:20 +00:00
5c985d4f81 [downloader] Let _ffmpeg_ handle DASH segments
Fixes https://github.com/ytdl-org/youtube-dl/issues/31792 after 3da1783.
2023-03-14 16:23:20 +00:00
8c86fd33dc [doc] Improve "guidance" on bug reporting 2023-03-09 16:40:30 +00:00
27d41d7365 [doc] Recommend "Get cookies.txt LOCALLY" extension in README.md (#31763)
* remove link to suspect "Get cookies.txt" extension, dropped from Chrome store
* link to new Manifest V3-compatible open-source "Get cookies.txt LOCALLY" extension.

Fixes #31465.
2023-03-07 15:49:31 +00:00
0402710227 [jsinterp] Fix regexp parsing and .replace[All] method
* For performance, make regexp object instantiation lazy
 * Other small performance improvements
2023-03-07 01:24:21 +00:00
3e92c60fcd [jsinterp] Handle Date at epoch 0
See yt-dlp/yt_dlp#6400
2023-03-03 15:02:15 +00:00
3da17834a4 [Youtube] Construct dash formats with range query
See yt-dlp/yt_dlp#6369
2023-03-03 15:02:15 +00:00
f7ce98a21e [YouTube] Support @owner format in uploader_id etc
* implement https://github.com/ytdl-org/youtube-dl/issues/31530#issuecomment-1435734719
* update affected tests
* misc clean-ups
2023-02-24 12:22:16 +00:00
e67e52a8f8 [test] Support test-case with volatile ID (eg live show)
Signalled by regexp ID value, eg: `'id': r're:[\da-zA-Z_-]{8,}'`
2023-02-24 12:22:16 +00:00
1d3751c3fe Escape URLs in sanitized_Request, not sanitize_url d2558234cf5dd12d6896eed5427b7dcdb3ab7b5a added escaping of URLs while sanitizing. However, sanitize_url may not always receive an actual URL. Eg: When using youtube-dl "search query" --default-search ytsearch, search query gets escaped to search%20query before being prefixed with ytsearch: which is not the intended behavior. So the escaping is moved to sanitized_Request instead. 2023-02-20 20:27:25 +00:00
df
6067451e43 [Vimeo] Fix e19ec52 for tween-age Pythons
* a check in older Pythons in the 2.7 and earlier, 3.3, 3.4 series caused "sre_constants.error: nothing to repeat"
* satisfy the check by avoiding nested qualifiers that can match empty string

Resolves #31597
2023-02-20 01:41:46 +00:00
57802e632f [jsinterp] Fix dict comprehension for Py2.6
Resolves #31600
2023-02-19 13:48:58 +00:00
2dd6c6edd8 [YouTube] Avoid crash if uploader_id extraction fails
See #31530.
2023-02-17 11:16:54 +00:00
dd9aa74bee [test] Avoid name TestIE which causes a pytest warning
See: 060ac76257
2023-02-14 16:36:40 +00:00
42b098dd79 [InfoExtractor] Handle unquoted values in OpenGraph searches 2023-02-14 02:53:16 +00:00
6f8c2635a5 [StreamsbIE] Add extractor for streamsb.com (viewsb.com) (#31517)
* Add extractor for streamsb.com (viewsb.com)

* make data url using app.js version

---------

Co-authored-by: dirkf <fieldhouse@gmx.net>
2023-02-13 03:54:51 +00:00
de48105dd8 [KommunetvIE] Add extractor for kommunetv.no (#31516)
* Add extractor for kommunetv.no
* Using utils.update_url instead of regex

---------

Co-authored-by: dirkf <fieldhouse@gmx.net>
2023-02-13 03:54:51 +00:00
822f19f05d [FileMoonIE] Add extractor for filemoon.sx (#31515)
---------

Co-authored-by: dirkf <fieldhouse@gmx.net>
2023-02-13 03:54:51 +00:00
33db85c571 [feat]: Add support to external downloader aria2p (#31500)
* feat: add class Aria2pFD

* feat: create call_downloader function

* feat: a colorful download interface to aria2pFD

* feat: change value name

* Apply suggestions from code review

Co-authored-by: dirkf <fieldhouse@gmx.net>

* Typo in suggestion

* fix: remove unused value

* fix: add not function to return value(0 is normal); add total_seconds to download.eta(timedelta object); add waiting status when hook progress

* fix: remove unuse method ..utils.format_bytes

* fix: be up to flake8

* fix: be up to flake8

* Apply suggestions from code review

* [feat] test external downloader aria2p

* [feat] test external downloader aria2p

* [fix] test_external_downloader.py

* Apply suggestions from code review

Co-authored-by: dirkf <fieldhouse@gmx.net>

* Apply suggestions from code review

Co-authored-by: dirkf <fieldhouse@gmx.net>

* Update test/test_external_downloader.py

Co-authored-by: dirkf <fieldhouse@gmx.net>

* Update test/test_external_downloader.py

Co-authored-by: dirkf <fieldhouse@gmx.net>

* Update youtube_dl/downloader/external.py

Co-authored-by: dirkf <fieldhouse@gmx.net>

* refactoring code and fix bugs

* Apply suggestions from code review

* Rename test_external_downloader.py to test_downloader_external.py

---------

Co-authored-by: dirkf <fieldhouse@gmx.net>
2023-02-13 03:54:51 +00:00
f33923cba7 [rbgtum] Add new extractor (#31305)
* [rbgtum] Add new extractor

* Small update, force CI

---------

Co-authored-by: dirkf <fieldhouse@gmx.net>
2023-02-13 03:54:51 +00:00
e8198c517b [YouTube] Fix tests 2023-02-13 03:54:51 +00:00
bafb6dec72 [YouTube] Refresh compat/utils usage
* import parse_qs()
* import parse_qs in lazy_extractors (clears old TODO)
* clean up old compiled lazy_extractors for Py2
* use update_url()
2023-02-13 03:54:51 +00:00
4e04f10499 [compat] Update test_compat
[skip ci]
2023-02-13 03:54:51 +00:00
90c9f789d9 [utils] Add parse_qs, update_url
[skip ci]
2023-02-13 03:54:51 +00:00
249f2b6316 [compat] Systematise compat_ naming
[skip ci]
2023-02-13 03:54:51 +00:00
d6b14ba316 [test] Fix TestAgeRestriction
* age restriction may cause DownloadError
* update obsolete test URLs
[skip ci]
2023-02-13 03:54:51 +00:00
30e986b834 [YouTube] Add signatureTimestamp for age-gate bypass 2023-02-13 03:54:51 +00:00
58988c1421 [YouTube] Bypass age-gating for certain restricted videos
* Use TVHTML5_SIMPLY_EMBEDDED_PLAYER client

* Also add and fix tests

* Introduce and use new utility function `update_url()`
2023-02-13 03:54:51 +00:00
e19ec52322 [Vimeo] Support /user{video_id}/{slug} URL format 2023-02-12 22:16:00 +00:00
f2f90887ca [Vimeo] Fix Unable to extract info section redux
* as reported in yt-dlp/yt-dlp#6149
* also allow newline in target JSON object
2023-02-12 22:16:00 +00:00
cd987e6fca [jsinterp] Nits 2023-02-12 22:16:00 +00:00
d947ffe8e3 [IGN] Overhaul extractor to avoid URL redirection loop
Consequently/also:
* centralise video data extraction
* detect 404 and 503 expected errors
* handle the test video in IGNVideo
* handle two additional page formats for the tests in IGNArticle
2023-02-12 22:16:00 +00:00
384f632e8a [ITV] Overhaul ITV extractor (#30266)
* support ITVX URLs (thanks Vangelis66)
* support legacy ITV Hub URLs
* include extraction fix 4c57dd2 from sleaux-meaux 3 May 2021
* include extraction fix 6fbcc16, fix by staubichsauger & pukkandan
* work-around duration parsing pending fix to utils.parse_duration
* apply default vanilla UA for pages and media to avoid site blocking
* also detect and report `Episode not found` instead of generic 404
* rework ITVBTCCIE with geo-block detection, best effort geo-restriction handling, news article support
* fix tests
2023-02-03 21:10:07 +00:00
9d17948b5a [myvideoge] Add new extractor (#31360)
NB download tests on CI servers blocked 

Co-authored-by: Alfonso Solbes <fonk666@gmail.com>
2023-02-02 23:25:44 +00:00
f316f5d4e3 [xhamster] add support for new domain xhvid.com (#31370) 2023-02-02 23:20:14 +00:00
bc6f94e459 [FIFA] Back-port extractor from yt-dlp (#31385) 2023-02-02 23:19:03 +00:00
be3392a0d4 [Blerp] Add new extractor (#31398)
Co-authored-by: dirkf <fieldhouse@gmx.net>
2023-02-02 17:33:09 +00:00
6d829d8119 [YouTube] Fix not finding videos listed under a channel's "shorts" subpage. (#31409)
Resolves #31336

Co-authored-by: Jouni Järvinen <rautamiekka@users.noreply.github.com>
Co-authored-by: dirkf <fieldhouse@gmx.net>
2023-02-02 17:26:31 +00:00
98b0cf1cd0 [Callin] Add new extractor (#31414)
Co-authored-by: dirkf <fieldhouse@gmx.net>
2023-02-02 17:21:05 +00:00
e9611a2a36 [pr0gramm] implement InfoExtractor, Resolves #31433 (#31434)
* [pr0gramm] implement infoextractor

* [pr0gramm] remove misplaced comment, uncapture regex-group

* [pr0gramm]: specify utf-8 coding

* [pr0gramm]: add trailing comma to lists for maintainability

* [pr0gramm]: ie only sets upload_date attribute

* [pr0gramm]: add video_id to title

* [pr0gramm]: more forgiving _valid_url regex

* [pr0gramm]: add uploader to title, if set

* Discriminate URL pattern

---------

Co-authored-by: dirkf <fieldhouse@gmx.net>
2023-02-02 17:13:39 +00:00
807e593a32 [cammodels] fix and improve extractor (#31453)
Co-authored-by: dirkf <fieldhouse@gmx.net>
2023-02-02 17:12:36 +00:00
297fbff23b [doc] Fixed typo appearing to promise an example (#31489)
Resolves #31425 

Co-authored-by: dirkf <fieldhouse@gmx.net>
2023-02-02 17:10:09 +00:00
37cbdfa0e7 [americastestkitchen] Add support for downloading entire series (#31493)
Also
* support new sites and URL patterns
* back-port from yt-dlp

Co-authored-by: dirkf <fieldhouse@gmx.net>
2023-02-02 16:58:21 +00:00
295736c9cb [jsinterp] Improve parsing
* support subset `... else if ...`
* support `while`
* add `RegExp` class
* generalise `new` support
* limited more debug strings
* matching test changes
2023-02-02 16:31:49 +00:00
14ef89a8da Support if statements
Fix for yt-dlp/yt_dlp#6131
Closes #31509
2023-02-02 13:12:46 +00:00
195f22f679 [generic] Improve KVS (etc) extraction 2022-11-13 15:09:29 +00:00
fc2beab0e7 [generic] Improve KVS (etc) extraction
* detect kt_player('kt_player', 'https://.../kt_player.swf?v=5...
* detect age limit if 18 USC 2257 is mentioned
* test with shooshtime.com

Partially resolves #31332.
2022-11-13 14:59:30 +00:00
1a4fbe8462 Added ThisVid.com support (#29187)
* add ThisVidIE, ThisVidMemberIE, ThisVidPlaylistIE
* redirect embed to main page for more metadata
* use KVS extraction newly added to GenericIE and remove duplicate tests
* also add MrDeepFake etc compat to GenericIE
(closes #22390)

Co-authored-by: dirkf <fieldhouse@gmx.net>
2022-11-13 13:22:04 +00:00
c2f9be3e63 [generic] Add KVS player extraction 2022-11-12 11:55:05 +00:00
604762a9f8 [common:jwplayer] Improve jwplayer extraction and parsing (#31000)
* don't crash parser if jwplayer_data is invalid (empty, or no formats)
* use `label` in `sources[n]` as `format_id`
* relax `jwplayer().setup(...)` RE (also rework PR #27274 enhancement)
* detect more manifest formats in _parse_jwplayer_formats() (from PR #29596)
* improve metadata extraction (from PR #25433)
* remember URLs in a set
* use parse_resolution() in format
* extract filesize in format (from yt-dlp)

Co-authored-by: kikuyan <kikuyan@users.noreply.github.com>
Co-authored-by: martin54 <martin54@users.noreply.github.com>
2022-11-11 00:49:13 +00:00
47e70fff8b [PeekVids, PlayVids] Add new extractor (#29765)
* Merge back-port from yt-dlp
* Merge features from PR #29798
* Improve metadata extraction

Co-authored-by: dirkf <fieldhouse@gmx.net>
Co-authored by: AXDOOMER
2022-11-09 20:26:30 +00:00
de39d1281c [extractor/ceskatelevize] Back-port extractor from yt-dlp, etc (#30713)
* back-port extractor, removing CeskaTelevizePoradyIE
* follow redirect URL
* support liveBroadcast and videobonusDetail in __NEXT__ data
* return single video for singleton playlist
* fix/add tests
2022-11-04 10:13:07 +00:00
27ed77aabb [utils] Backport traverse_obj (etc) from yt-dlp (#31156)
* Backport traverse_obj and closely related function from yt-dlp (code by pukkandan)
* Backport LazyList, variadic(), try_call (code by pukkandan)
* Recast using yt-dlp's newer traverse_obj() implementation and tests (code by grub4k)
* Add tests for Unicode case folding support matching Py3.5+ (requires f102e3d)
* Improve/add tests for variadic, try_call, join_nonempty

Co-authored-by: dirkf <fieldhouse@gmx.net>
2022-11-03 10:09:37 +00:00
c4b19a8816 [compat] Work around in case folding for narrow Python build
Resolves #31324.
2022-11-02 11:56:26 +00:00
087ddc2371 [compat] Add test for compat_casefold() 2022-11-01 22:47:02 +00:00
65ccb0dd4e [compat] Add test for compat_casefold() 2022-11-01 21:33:39 +00:00
a874871801 [compat] Reformat casefold.py for easier updating 2022-11-01 19:25:59 +00:00
b7c25959f0 [compat] Unify unicode/str compat and move up 2022-11-01 12:40:23 +00:00
f102e3dc4e [compat] Add compat_casefold and compat_re_Match, for traverse_obj() port 2022-10-31 21:27:14 +00:00
a19855f0f5 [compat] Add Python 2 Unicode casefold using a trivial wrapper around icu/CaseFolding.txt 2022-10-31 21:18:36 +00:00
ce5d36486e [netease] Support urls shared from mobile app (#31304)
Co-authored-by: dirkf <fieldhouse@gmx.net>
2022-10-30 11:48:44 +00:00
d25cf62086 [netease] Impove error handling (#31303)
* add warnings for users outside of China
* skip empty song urls

Co-authored-by: dirkf <fieldhouse@gmx.net>
2022-10-30 11:46:46 +00:00
502cefa41f [Vimeo] Update variable name in hydration JSON pattern
Fixes #31311
2022-10-27 14:33:00 +00:00
0faa45d6c0 [BongaCams] Support new .net domain
Resolves #31262.
2022-10-20 11:06:44 +00:00
447edc48e6 Fix ADN extractor (#31275)
* Rename Anime Digital Network to Animation Digital Network, animationdigitalnetwork.fr
* Update the test to an available video
* Update the decoding key of subtitles
* Keep the support of old URLs
* Add a test to match the old URL
* Reduce redundancy of the URL name
* Fix md5 ^^"
* Fix undefined _BASE
* Process HTTP error text (eg geo-block) correctly and uniformly in Py3, Py2
* Skip test for CI since geo-blocked

Signed-off-by: ache <ache@ache.one>
Co-authored-by: dirkf <fieldhouse@gmx.net>
2022-10-18 16:06:27 +01:00
ee8560d01e [ManyVids] Support new single-page app structure 2022-10-13 02:42:49 +00:00
7135277fec [ManyVids] Support new single-page app structure
See https://github.com/yt-dlp/yt-dlp/issues/5210#issuecomment-1276919962.
2022-10-13 01:59:01 +00:00
7bbd5b13d4 [Motherless] Pull from yt-dlp, etc
* use username field
* loosen regexes
* warn on page count 0 in group
* avoid reloading group page 1
Closes #29626
2022-10-12 01:09:55 +01:00
c91cbf6072 [netease] Get netease music download url through player api (#31235)
* remove unplayable song from test
* compatible with python 2
* using standard User_Agent, fix imports
* use hash instead of long description
* fix lint
* fix hash
2022-10-11 13:55:09 +01:00
11b284c81f [Common:JWPlayer] Fix x1000 scaling error
See https://github.com/yt-dlp/yt-dlp/issues/5106#issuecomment-1264625161
2022-10-11 12:36:44 +00:00
c94a459a24 [utils] Sanitize look-alike Unicode glyphs in non-ID filename fields when --restrict-filenames
Implements https://github.com/ytdl-org/youtube-dl/issues/31216#issuecomment-1236102822, which has a test.
2022-10-11 12:18:12 +00:00
6e2626f092 [JSInterp] Improve separation logic
Based on 0468a3b325
2022-10-11 05:58:10 +01:00
c282e5f8d7 [ZDF] Overhaul ZDF extractors
* pull some yt-dlp changes into ZDFBaseIE._extract_format()
* add test cases from yt-dlp to ZDFIE
* fix crash in ZDFIE._extract_mobile() when object had no `formitaeten`
* improve title extraction in ZDFChannelIE (remove trailing station ident)
* avoid extracting non-video playlist items (fixes #31149)
2022-10-11 00:05:17 +01:00
2ced5a7912 [test] Implement string "lambda x: condition(x)" as an expected value
Semantics equivalent to `assert condition(got)`
2022-10-11 00:05:17 +01:00
82e4eca711 [motherless] Fixed the broken uploader_id in the extractor (#31243)
* Fixed the broken uploader_id in the extractor.
* Make uploader_id RE looser
* Fix uploader_id in test Motherless_3
* Fix group pagination
* # coding: utf-8

Co-authored-by: Andy Xuming <xuminic@gmail.com>
Co-authored-by: dirkf <fieldhouse@gmx.net>
2022-10-10 23:52:48 +01:00
1b1442887e [manyvids] Improve extraction (#31172)
* extract all formats from page
* extract description, uploader, views, likes
* downrate previews
* fix tests
* use txt_or_none()
2022-10-10 19:26:32 +01:00
22127b271c [NRK] Remove explicit Accept-Encoding header that invites Brotli
Fixes #31285
2022-10-10 17:41:40 +00:00
d35557a75d [Telegraaf] Use mobile GraphQL API endpoint
Workaround for Cloudflare 403
Fixes https://github.com/yt-dlp/yt-dlp/issues/5000
Authored by: coletdjnz
2022-10-04 11:43:08 +01:00
9493ffdb8b [test] Use windows-2019 for tests
(At least for now) resolves #31249
2022-10-04 11:31:29 +01:00
7009bb9f31 [jsinterp] Workaround operator associativity issue
* temporary fix for player 5a3b6271 [1]

1. https://github.com/yt-dlp/yt-dlp/issues/4635#issuecomment-1235384480
2022-09-03 00:53:56 +01:00
218c423bc0 [cache] Add cache validation by program version, based on yt-dlp 2022-09-01 13:28:30 +01:00
55c823634d [jsinterp] Handle new YT players 113ca41c, c57c113c
* add NaN
* allow any white-space character for `after_op`
* align with yt-dlp f26af78a8ac11d9d617ed31ea5282cfaa5bcbcfa (charcodeAt and bitwise overflow)
* allow escaping in regex, fixing player c57c113c
2022-09-01 10:57:12 +01:00
4050e10a4c [options] Document that postprocessing is not forced by --postprocessor-args
Resolves #30307
2022-08-29 13:02:17 +01:00
ed5c44e7b7 [compat] Replace deficient ChainMap class in Py3.3 and earlier
* fix version check
2022-08-26 12:22:01 +01:00
0f6422590e [compat] Replace deficient ChainMap class in Py3.3 and earlier 2022-08-26 10:24:42 +01:00
4c6fba3765 [jsinterp] Improve try/catch/finally support 2022-08-26 08:51:17 +01:00
d619dd712f [jsinterp] Fix bug in operator precedence
* from 164b03c486
* added tests
2022-08-25 12:16:10 +01:00
573b13410e [YouTube] Improve error check for n-sig processing 2022-08-25 12:14:59 +01:00
66e58dccc2 [core] Avoid processing empty format list after removing bad formats
* also ensure compat encoding of error strings
2022-08-21 00:45:06 +01:00
556862bc91 [utils] Ensure RFC3986 encoding result is unicode 2022-08-21 00:45:06 +01:00
a8d5316aaf [infoq] Avoid crash if the page has no mp3Form
* proposed fix for issue #31131, aligns with yt-dlp

Co-authored-by: dirkf <fieldhouse@gmx.net>
2022-08-19 21:00:21 +01:00
fd3f3bebd0 [uktvplay] Support domain without .uktv 2022-08-19 19:11:08 +01:00
46b8ae2f52 [jsinterp] Clean up and pull yt-dlp style
* add compat_re_Pattern
* improve compat_collections_chain_map
* use class JS_Undefined
* remove unused code
2022-08-19 15:34:33 +01:00
538ec65ba7 [jsinterp] Handle regexp literals and throw/catch execution (#31182)
* based on f6ca640b12, thanks pukkandan
* adds parse support for regexp flags
2022-08-19 11:45:04 +01:00
b0a60ce203 [jsinterp] Improve JS language support (#31175)
* operator ??
* operator ?.
* operator **
* accurate operator functions
* `undefined` handling
* object literals {a: 1, "b": expr}
* more tests for weird JS comparisons: see https://github.com/ytdl-org/youtube-dl/issues/31173#issuecomment-1217854397.
2022-08-17 14:22:02 +01:00
e52e8b8111 [postprocessor] Don't replace existing value with null metadata parsed from title 2022-08-15 16:45:04 +01:00
d231b56717 [jsinterp] Overhaul JSInterp to handle new YT players 4c3f79c5, 324f67b9 (#31170)
* back-port from yt-dlp 8f53dc44a0cc1c2d98c35740b9293462c080f5d0, thanks pukkandan
* also support void, improve <</>> precedence, improve expressions in comma-list
* add more tests
2022-08-14 18:45:45 +01:00
e6a836d54c [core] Make --max-downloads ... stop immediately on reaching the limit
Based on and closes #26638.
2022-08-10 15:37:59 +01:00
deee741fb1 [test, etc] Improve download test logs; also clean up some new flake8 issues (#31153)
* [test] Identify testcase errors better
* [test] Identify download errors better
* [extractor/minds] Linter
* [extractor/aes] Linter
2022-08-09 21:05:00 +01:00
Wes
adb5294177 [aenetworks] Update _THEPLATFORM_KEY and _THEPLATFORM_SECRET (#29749)
Fixes ytdl-org/youtube-dl#29300
2022-07-30 02:10:00 +01:00
5f5c127ece [VVVVID] Support video/dash types (#31060)
Resolves #31030.
2022-07-12 00:35:40 +01:00
090acd58c1 [options] Improve be35e53 (--match-/reject-title parameter value)
Resolves #31064.
2022-07-03 20:05:21 +01:00
a03b9775d5 [Mediaset] Support player version number in URL pattern
Ref: https://github.com/yt-dlp/yt-dlp/issues/4141
2022-06-26 14:24:06 +01:00
8a158a936c [NHK] Use new API URL 2022-06-15 18:28:19 +01:00
11665dd236 [test] Fix linter for 3aa94d7945 2022-06-15 18:28:19 +01:00
cc179df346 [XHamster] Support xhday.com alias, extract uploader_id
* support xhday.com alias for xhamster.com (resolves #31023)
  Authored by: dirkf
* extract `uploader_id`:
  from 908b56eaf7
  (PR https://github.com/yt-dlp/yt-dlp/pull/844)
  Authored by: octotherp
2022-06-12 14:10:38 +01:00
0700fde640 [utils, etc] Kill child processes when yt-dl is killed
* derived from PR #26592, closes #26592

Authored by: Unrud
2022-06-10 19:57:46 +01:00
811c480f7b [YouTube] Support JSON3 subtitle format
* subtitle tests updated to match
2022-06-09 15:25:23 +01:00
3aa94d7945 [test] Fix workable subtitle tests (except YT) and mark others as skip, broken
* broken tests need to be fixed when fixing the respective IE
2022-06-08 23:11:33 +01:00
ef044be34b [test] Skip not _WORKING IE in subtitle tests; use unittest.skipTest throughout 2022-06-08 15:52:21 +01:00
530f4582d0 [HRFernsehen] Back-port new extractor from yt-dlp
Closes #26445, where this was originally proposed.
2022-06-06 19:29:48 +01:00
1baa0f5f66 [utils] Escape URL while sanitizing
Closes #31008, #yt-dlp/263

While this fixes the issue in question, it does not try to address the root-cause of the problem
Refer: 915f911e365736227e134ad654601443dbfd7ccb, f5fa042c82300218a2d07b95dd6b9c0756745db3
2022-06-06 16:03:04 +01:00
9aa8e5340f [Readme] Clarified extractor naming (#29799)
* Exported usable extractors must be named `xxxxIE`

Co-authored-by: dirkf <fieldhouse@gmx.net>
2022-05-30 02:50:50 +01:00
04fd3289d3 [YouPorn] Improve upload_date extraction
See https://github.com/yt-dlp/yt-dlp/issues/2701#issuecomment-1034341883
2022-05-28 13:54:32 +01:00
52c3751df7 [utils] Enable ALPN in HTTPS to satisfy broken servers
See https://github.com/yt-dlp/yt-dlp/issues/3878
2022-05-28 13:52:51 +01:00
187a48aee2 [YouTube] Handle player c5a4daa1 with indirect n-function definition
* resolves #30976
2022-05-24 15:43:56 +01:00
be35e5343a Update options.py 2022-05-20 05:25:54 +01:00
c3deca86ae [wat.tv] Add version pver to metadata API call
Resolves #30959.
2022-05-19 17:41:48 +00:00
c7965b9fc2 [NHK] Support alphabetic characters in 7-char NhkVod IDs (#29682) 2022-05-09 18:54:41 +01:00
e988fa4523 [doc] Clarify test naming 2022-04-29 16:56:00 +01:00
e27d8d819f [streamcz] Remove empty '{}'.format() for Py2.6
Use `'-join()'` here, or `{0}`, ..., in general.
2022-04-29 13:36:02 +01:00
ebc627847c [KTH] Add new extractor for KTH play (#30885)
* Implement extractor for KTH play
* Make KTH Play url regex more relaxed
2022-04-28 10:18:10 +01:00
a0068bd6be [Youtube] Fix "n" descrambling for player fae06c11
Resolves #30856.
2022-04-15 16:07:09 +01:00
b764dbe773 Disable blank issues 2022-04-10 05:49:09 +01:00
871645a4a4 [RAI] Fix extraction of http formats
From https://github.com/yt-dlp/yt-dlp/pull/3272
Closes https://github.com/yt-dlp/yt-dlp/issues/3270
Authored by: nixxo
2022-04-05 15:21:59 +01:00
1f50a07771 [RAI] Extend formats with direct http mp4 link (PR #27990)
* initial support for creating direct mp4 link
* improved regexes and info extraction
* added "connection: close" to request headers
* updated to https://github.com/yt-dlp/yt-dlp/pull/208
2022-04-05 15:21:59 +01:00
9e5ca66f16 [RAI] Added checks for DRM protected content (PR #27657)
reviewed by pukkandan (https://github.com/yt-dlp/yt-dlp/pull/150)
2022-04-05 15:21:59 +01:00
17d295a1ec [extractor/bilibili] Fix path "/audio/auxxxxx" download return 403 2022-04-01 00:46:34 +01:00
49c5293014 Ignore --external-downloader-args if --external-downloader was rejected
... and generate warning
2022-03-25 14:47:26 +00:00
df
6508688e88 Make default upload_/release_date a compat_str
Ensures download tests pass in Python 2 as well as 3; also
add YoutubeDL tests for timestamp -> upload_date etc.
2022-02-26 10:29:42 +00:00
4194d253c0 Avoid skipping ID when unlisted_hash is numeric
Pattern needed a non-greedy match; also replaced a redundant test with one for this, issue 29690
2022-02-26 10:29:42 +00:00
f8e543c906 [Alsace20TV] Add new extractors Alsace20TVIE, Alsace20TVEmbedIE 2022-02-24 18:43:47 +00:00
c4d1738316 [CPAC] Add extractor for Canadian Parliament
CPACIE: single episode
CPACPlaylistIE: playlists and searches
2022-02-24 18:27:57 +00:00
1f13ccfd7f Fixed groups() call on potentially empty regex search object (#30676)
* Fixed groups() call on potentially empty regex search object.
- https://github.com/ytdl-org/youtube-dl/issues/30521

* minimising lines changed

Co-authored-by: yayorbitgum <50963144+yayorbitgum@users.noreply.github.com>
2022-02-24 18:26:58 +00:00
923292ba64 [aliexpress] Fix test case 2022-02-24 13:44:52 +00:00
782bfd26db [bigo] add support for bigo.tv (#30635)
* [bigo] add support for bigo.tv

* [bigo] prepend "Bigo says"

* title fallback

* add error for invalid json data
2022-02-24 13:34:32 +00:00
3472227074 [rutv] fix vbr for empty string value (#30623)
* [rutv] use str_to_int() (thx dirkf)
2022-02-14 17:54:31 +00:00
bf23bc0489 add missing __future__ import unicode_literals 2022-02-14 07:07:05 +00:00
85bf26c1d0 resolve problem with unpacking operator for <py3.5 2022-02-14 07:07:05 +00:00
d8adca1b66 [streamcz] test fixes and one additional test 2022-02-14 07:07:05 +00:00
d02064218b do not use f-strings 2022-02-14 07:07:05 +00:00
b1297308fb avoid traverse_obj function 2022-02-14 07:07:05 +00:00
8088ce036a revert: use _match_valid_url function 2022-02-14 07:07:05 +00:00
29f7bfc4d7 [streamcz] cherry-pick from yt-dlp
Cherry-picked-from: 7d449fff5346 ("[streamcz] Fix extractor (#1616)")
2022-02-14 07:07:05 +00:00
74f8cc48af [extractor/videa] Back-port from yt-dlp PRs 463+1028
Authored by: nyuszika7h
2022-02-11 12:43:26 +00:00
8ff961d10f [extractor/videa] fix extraction in Py2
Fixes #30416
2022-02-11 12:43:26 +00:00
266b6ef185 [BBC] Also allow PID with leading 'l' (live?) 2022-02-09 21:21:59 +00:00
825d3426c5 [Nuvid] Use site JSON for video details (#29332)
Back-port yt-dlp PR 1022 onto PR #17890 and update

Video details aren't in the original HTML now but populated by async JS

Co-authored by: u-spec-png
Co-authored by: vidaritos
2022-02-09 02:40:34 +00:00
47b0c8697a [ARD] Back-port subtitle extraction from yt-dlp PR 2409
Authored by: fstirlitz
Fixes #30543
Closes #17766 (thanks ngdio)
2022-02-07 13:47:38 +00:00
734dfbb4e3 Remove redundant assigning format_id 2022-02-05 03:04:35 +00:00
df
ddc080a562 Add ArteTVCategoryIE to support category playlists 2022-02-05 03:02:56 +00:00
16a3fe2ba6 Updated Album URL regex
Mistakenly forgot to edit a line in last commit.

Co-authored-by: dirkf <fieldhouse@gmx.net>
2022-02-05 02:53:23 +00:00
c820a284a2 [extractor/audiomack] Updated URL regex, corrected invalid testcases, fixed bug
Co-authored-by: dirkf <fieldhouse@gmx.net>
2022-02-05 02:53:23 +00:00
58babe9af7 Support __INITIAL_DATA__ with stringified JSON
Add test and fix test for bbcthreeConfig
2022-02-05 02:51:46 +00:00
df
6d4932f023 Try for timestamp, description from window.__INITIAL_DATA__ pages 2022-02-05 02:51:46 +00:00
92d73ef393 [niconico] Implement heartbeat for download 2022-02-05 02:47:21 +00:00
91278f4b6b [niconico] Back-port extractor from yt-dlp
Add Nico search extractors, fix extraction
2022-02-05 02:47:21 +00:00
73e1ab6125 [test:download] Only extract enough videos for playlist_mincount 2022-02-05 02:47:21 +00:00
584715a803 [applepodcasts] Extract default thumbnail image 2022-02-05 02:32:45 +00:00
e00b0eab1e [applepodcasts] Improve format extraction
Set acodec and vcodec, etc, to avoid breaking, eg, bestaudio
2022-02-05 02:32:45 +00:00
005339d637 [applepodcasts] Support new AMP-ish page structure 2022-02-05 02:32:45 +00:00
23ad6402a6 xvideos: Fix for #30271 2022-02-05 02:24:51 +00:00
9642344965 Fix tests for working IEs; disable obsolete WDRMobile 2022-02-05 02:22:45 +00:00
568c7005d5 Fix WDRMaus; extend URL matching for other Maus pages; improve ID extraction 2022-02-05 02:22:45 +00:00
5cb4833f40 Update URPlayIE extractor for Next.js page format, with subtitles 2022-02-05 02:16:53 +00:00
5197336de6 Support more deeply nested ptmd_path with test, update tests 2022-02-05 02:14:35 +00:00
01824d275b Additional tweaks: allow any .ndr.de, simplify quote match 2022-02-05 02:12:44 +00:00
39a98b09a2 Fix NDR, NJoy tests 2022-02-05 02:12:44 +00:00
f0a05a55c2 NJoy: improve extraction of NDR id, description, etc with current page formats 2022-02-05 02:12:44 +00:00
4186e81777 NDR: improve extraction of NDR id, description, etc with current page formats 2022-02-05 02:12:44 +00:00
b494824286 Support Tele5 pages with Discovery Networks format instead of JWPlatform 2022-02-05 02:08:11 +00:00
8248133e5e Back-port yt-dlp Viki extractor
From https://github.com/yt-dlp/yt-dlp/pull/2540
2022-02-04 15:49:12 +00:00
27dbf6f0ab Return the item itself if playlist has one entry
Removes playlist spam from log
2022-02-04 14:28:50 +00:00
61d791726f Find TV2DK Kaltura ID in Nuxt.js page format 2022-02-04 14:28:50 +00:00
0c0876f790 [youtube:search] Add tests 2022-02-04 11:09:18 +00:00
7a497f1405 Rework 2c2c2bd with an actual Mix page and realistic playlist size
From 2c2c2bd348 (commitcomment-65953545)
2022-02-04 04:09:23 +00:00
5add3f4373 Merge branch 'pukkandan-yt-searchurl' into yt-dl-master
Closes #27749
2022-02-04 03:50:32 +00:00
78ce962f4f [youtube] Support channel search
Code from cd684175ad
2022-02-03 01:02:58 +00:00
41f0043983 Avoid crashing if n-sig decode fails 2022-02-02 14:25:03 +00:00
34c06b16f5 Support Youtube Shorts URL format 2022-02-01 14:40:20 +00:00
1e677567cd [YouTube] Fix n-sig for player e06dea74 (#30582)
From yt-dl commit 48416bc
2022-02-01 14:39:03 +00:00
df
af9e72507e Implement n-param descrambling using JSInterp
Fixes #29326, closes #29790, closes #30004, closes #30024, closes #30052,
closes #30088, closes #30097, closes #30102, closes #30109, closes #30119,
closes #30125, closes #30128, closes #30162, closes #30173, closes #30186,
closes #30192, closes #30221, closes #30239, closes #30539, closes #30552.
2022-01-31 00:19:58 +00:00
6ca7b77696 Refactor JSInterpreter._separate
yt-dlp/yt-dlp/@06dfe0a, improve _MATCHING_PARENS
2022-01-30 00:05:54 +00:00
9d142109f4 Back-port test_youtube_signature.py from yt-dlp and fix JSInterp accordingly 2022-01-30 00:05:54 +00:00
1ca673bd98 Fix splice to handle float
Needed for new youtube js player f1ca6900
Add 57dbe8077f (diff-729b57caa8d006426f6a8960c061f519a8b6658682284015e069745af52ffb07)
2022-01-30 00:05:54 +00:00
df
e1eae16b56 Handle default in switch better
Add a1fc7ca074
Thanks coletdjnz
2022-01-30 00:05:54 +00:00
df
96f87aaa3b Back-port JS interpreter upgrade from yt-dlp PR #1437 2022-01-30 00:05:54 +00:00
df
5f5de51a49 Add compat_map/filter and use the former 2022-01-30 00:05:36 +00:00
df
39ca35e765 Fix test_youtube_flat_playlist_extraction 2022-01-29 20:00:21 +00:00
df
d76d59d99d Remove obsolete non-working test_youtube_toptracks 2022-01-29 20:00:21 +00:00
df
2c2c2bd348 Fix test_youtube_mix 2022-01-29 20:00:21 +00:00
df
46e0a729b2 Remove obsolete test_youtube_course 2022-01-29 20:00:21 +00:00
df
57044eaceb Fix test_youtube_playlist_noplaylist 2022-01-29 20:00:21 +00:00
a3373da70c Merge branch 'UP/youtube-dl' into dl/YoutubeSearchURLIE 2022-01-30 01:07:28 +05:30
2c4cb134a9 Fix max_results 2022-01-30 00:54:22 +05:30
bfe72723d8 Use itertools.islice 2022-01-30 00:49:55 +05:30
ed99d68bdd Add back YoutubeSearchURLIE 2022-01-30 00:41:47 +05:30
5014bd67c2 release 2021.12.17 2021-12-17 01:49:07 +07:00
e418823350 [ChangeLog] Actualize
[ci skip]
2021-12-17 01:43:16 +07:00
b5242da7d2 [youtube] Update signature function patterns (closes #30363) (#30366) 2021-12-17 01:42:17 +07:00
a803582717 [peertube] only call description endpoint if necessary (#29383) 2021-07-01 06:53:22 +00:00
7fb9564420 [periscope] pass referer to HLS requests(closes #29419) 2021-06-28 20:08:39 +01:00
379f52a495 [liveleak] Remove extractor (closes #17625, closes #24222) (#29331) 2021-06-21 04:23:50 +07:00
cb668eb973 [pornhub] Add support for pornhubthbh7ap3u.onion 2021-06-21 04:08:15 +07:00
751c9ae39a [pornhub] Detect geo restriction 2021-06-21 03:33:43 +07:00
da32828208 [pornhub] Dismiss tbr extracted from download URLs (closes #28927)
No longer reliable
2021-06-21 03:22:37 +07:00
2ccee8db74 [curiositystream:collection] Extend _VALID_URL (closes #26326, closes #29117) 2021-06-21 01:54:52 +07:00
47f2f2fbe9 [youtube] Make get_video_info processing more robust (closes #29333) 2021-06-21 01:35:21 +07:00
03ab02730f [youtube] Workaround for get_video_info request (refs #29333)
See https://github.com/ytdl-org/youtube-dl/issues/29333#issuecomment-864049544
2021-06-21 01:34:27 +07:00
4c77a2e538 [bilibili] Strip uploader name (#29202) 2021-06-21 01:03:21 +07:00
4131703001 [youtube] Update invidious instance list (#29281) 2021-06-21 00:42:09 +07:00
cc21aebe90 [umg:de] Update GraphQL API URL (#29304)
Previous one no longer resolves

Co-authored-by: Sergey M. <dstftw@gmail.com>
2021-06-21 00:41:14 +07:00
57b9a4b4c6 [nrk] Switch psapi URL to https (closes #29344)
Catalog calls no longer work via http
2021-06-21 00:36:28 +07:00
3a7ef27cf3 [postprocessor/ffmpeg] Show ffmpeg output on error (refs #22680) (#29336) 2021-06-20 23:58:19 +07:00
a7f61feab2 [egghead] Add support for app.egghead.io (closes #28404) (#29303)
Co-authored-by: Sergey M. <dstftw@gmail.com>
2021-06-17 10:34:33 +07:00
8fe5d54eb7 [appleconnect] Fix extraction (#29208) 2021-06-17 04:12:13 +07:00
d156bc8d59 [orf:tvthek] Add support for MPD formats (closes #28672) (#29236) 2021-06-17 04:02:06 +07:00
c2350cac24 [README.md] Update MSVC 2010 redist URL (closes #29222) 2021-06-06 05:32:27 +07:00
b224cf39d5 release 2021.06.06 2021-06-06 01:38:22 +07:00
5f85eb820c [ChangeLog] Actualize
[ci skip]
2021-06-06 01:32:15 +07:00
bb7ac1ed66 [facebook] Improve login required detection 2021-06-06 01:16:43 +07:00
fdf91c52a8 [youporn] Fix formats and view count extraction (closes #29216) 2021-06-06 00:11:09 +07:00
943070af4a [orf:tvthek] Fix thumbnails extraction (closes #29217) 2021-06-05 23:42:25 +07:00
82f3993ba3 [formula1] fix extraction(closes #29206) 2021-06-04 17:51:44 +01:00
d495292852 [ard] Relax _VALID_URL and fix video ids (closes #22724, closes #29091) 2021-05-30 06:14:59 +07:00
2ee6c7f110 [ustream] Detect https embeds (closes #29133) 2021-05-30 03:43:59 +07:00
6511b8e8d7 [ted] Prefer own formats over external sources (closes #29142) 2021-05-30 03:05:22 +07:00
f3cd1d9cec [twitch:clips] Improve extraction (closes #29149) 2021-05-30 01:49:51 +07:00
e13a01061d [twitch:clips] Add access token query to download URLs (closes #29136) 2021-05-30 01:47:33 +07:00
24297a42ef [youtube] Fix get_video_info request (closes #29086, closes #29165) 2021-05-30 00:36:26 +07:00
1980ff4550 [vimeo] fix vimeo pro embed extraction(closes #29126) 2021-05-26 11:04:39 +01:00
dfbbe2902f [redbulltv] fix embed data extraction(closes #28770) 2021-05-17 12:56:49 +01:00
e1a9d0ef78 [shahid] relax _VALID_URL(closes #28772, closes #28930) 2021-05-17 12:37:39 +01:00
f47627a1c9 release 2021.05.16 2021-05-16 22:55:05 +07:00
efeb9e0fbf [ChangeLog] Actualize
[ci skip]
2021-05-16 22:40:39 +07:00
e90a890f01 [playstuff] Add extractor (closes #28901, closes #28931) 2021-05-16 22:31:37 +07:00
199c645bee [eroprofile] Skip test 2021-05-16 22:01:51 +07:00
503a3744ad [eroprofile] Fix extraction (closes #23200, closes #23626, closes #29008) 2021-05-16 21:57:21 +07:00
ef03721f47 [vivo] Add support for vivo.st (#29009)
Co-authored-by: Sergey M. <dstftw@gmail.com>
2021-05-16 21:46:32 +07:00
1e8aaa1d15 [generic] Add support for og:audio (closes #28311, closes #29015) 2021-05-16 21:42:38 +07:00
6423d7054e [options] Fix thumbnail option group name (closes #29042) 2021-05-16 21:34:10 +07:00
eb5080286a [phoenix] Fix extraction (closes #29057) 2021-05-16 21:21:14 +07:00
286e01ce30 [generic] Add support for sibnet embeds 2021-05-16 20:50:32 +07:00
8536dcafd8 [vk] Add support for sibnet embeds (closes #9500) 2021-05-16 20:48:24 +07:00
552b139911 [generic] Add Referer header for direct videojs download URLs (closes #2879, closes #20217, closes #29053) 2021-05-16 20:29:35 +07:00
2202cef0e4 [orf:radio] Switch download URLs to HTTPS (closes #29012) (#29046) 2021-05-16 19:54:15 +07:00
a726009987 [blinkx] Remove extractor (closes #28941)
No longer exists.
2021-05-05 04:12:35 +07:00
03afef7538 [medaltv] Relax _VALID_URL (#28884)
Co-authored-by: Sergey M. <dstftw@gmail.com>
2021-05-05 03:44:07 +07:00
b797c1cc75 [YoutubeDL] Improve extract_info doc (#28946)
Co-authored-by: Sergey M. <dstftw@gmail.com>
2021-05-05 03:31:24 +07:00
04be55307a [funimation] Add support for optional lang code in URLs (closes #28950) 2021-05-05 02:54:12 +07:00
504e4d804d [gdcvault] Add support for HTML5 videos 2021-05-05 02:44:29 +07:00
1786cd3fe4 [dispeak] DRY and update tests (closes #28970) 2021-05-05 02:30:42 +07:00
b8645c1f58 [dispeak] Improve FLV extraction (closes #13513) 2021-05-05 02:24:55 +07:00
fe05191b8c [kaltura] Improve iframe extraction (#28969)
Co-authored-by: Sergey M. <dstftw@gmail.com>
2021-05-05 02:14:35 +07:00
0204838163 [kaltura] Make embed code alternatives actually work 2021-05-05 02:01:22 +07:00
a0df8a0617 [cda] Improve extraction (closes #28709, closes #28937) 2021-05-01 22:53:30 +07:00
d1b9a5e2ef [twitter] Improve formats extraction from vmap URL (closes #28909) 2021-05-01 19:00:39 +07:00
ff04d43c46 [xtube] Fix formats extraction (closes #28870) 2021-05-01 18:33:05 +07:00
d2f72c40db [svtplay] Improve extraction (closes #28507, closes #28876) 2021-05-01 18:09:32 +07:00
e33dfb445c [tv2dk] Fix extraction (closes #28888) 2021-05-01 17:53:27 +07:00
94520568b3 [workflows/ci.yml] Update link to jython-installer 2021-04-26 02:16:47 +07:00
273964d190 release 2021.04.26 2021-04-26 01:33:30 +07:00
346dd3b5e8 [ChangeLog] Actualize
[ci skip]
2021-04-26 01:29:50 +07:00
f5c2c06231 [xfileshare] Add support for wolfstream.tv (#28858) 2021-04-26 00:32:47 +07:00
57eaaff5cf [francetvinfo] Improve video id extraction (closes #28792) 2021-04-25 22:52:28 +07:00
999329cf6b [workflows/ci.yml] Fix install nose for Jython 2021-04-24 23:52:16 +07:00
c6ab792990 [medaltv] Fix extraction (#28807)
numeric clip ids are no longer used by medal, and integer user ids are now sent as strings.
2021-04-24 19:10:35 +07:00
0db79d8181 [tver] Redirect all downloads to Brightcove (#28849) 2021-04-24 18:58:03 +07:00
7e8b3f9439 [youtube] Remove unused code 2021-04-21 05:37:51 +07:00
ac19c3ac80 [go] Improve video id extraction (closes #25207, closes #25216, closes #26058) 2021-04-21 05:35:39 +07:00
c4a451bcdd [test_execution] Add test for lazy extractors (refs #28780) 2021-04-21 04:47:29 +07:00
5ad69d3d0e [test_youtube_misc] Move YoutubeIE.extract_id test into separate module 2021-04-21 04:45:13 +07:00
32290307a4 [youtube] Fix lazy extractors (closes #28780) 2021-04-21 03:56:04 +07:00
dab83a2597 [bbc] Extract full description from __INITIAL_DATA__ (refs #28774) 2021-04-21 03:00:56 +07:00
41920fc80e [bbc] Extract description and timestamp from __INITIAL_DATA__ (#28774) 2021-04-21 02:51:55 +07:00
9f6c03a006 [cbsnews] Fix extraction for python <3.6 (closes #23359) 2021-04-17 05:05:31 +07:00
596b26606c release 2021.04.17 2021-04-17 03:50:09 +07:00
f20b505b46 [ChangeLog] Actualize
[ci skip]
2021-04-17 03:47:00 +07:00
cfee2dfe83 [utils] PEP 8 2021-04-17 03:32:04 +07:00
30a3a4c70f [lbry] Add support for HLS videos (closes #27877, closes #28768) 2021-04-17 03:23:47 +07:00
a00a7e0cad [utils] Add support for support for experimental HTTP response status code 308 Permanent Redirect (refs #27877, refs #28768) 2021-04-17 03:22:13 +07:00
54558e0baa [youtube] Improve stretch extraction and fix stretched ratio calculation (closes #28769) 2021-04-17 02:27:54 +07:00
7c52395479 [youtube:tab] Improve grid extraction (closes #28725) 2021-04-17 01:13:10 +07:00
ea87ed8394 [youtube:tab] Detect series playlist on playlists page (closes #28723) 2021-04-17 01:13:10 +07:00
d01e261a15 [youtube] Add more invidious instances (#28706) 2021-04-17 00:31:34 +07:00
79e4ccfc4b [pluralsight] Extend anti-throttling timeout (#28712) 2021-04-17 00:30:10 +07:00
06159135ef [youtube] Improve URL to extractor routing (closes #27572, closes #28335, closes #28742) 2021-04-17 00:07:32 +07:00
4fb25ff5a3 [maoritv] Add new extractor(closes #24552) 2021-04-09 09:02:37 +01:00
1b0a13f33c [youtube:tab] Pass innertube context and x-goog-visitor-id header along with continuation requests (closes #28702) 2021-04-09 02:10:34 +07:00
27e5a4464d [mtv] Fix Viacom A/B Testing Video Player extraction(closes #28703) 2021-04-08 18:54:44 +01:00
545d6cb9d0 [pornhub] Extract DASH and HLS formats from get_media end point (closes #28698) 2021-04-08 15:32:59 +07:00
006eea564d [cbssports] fix extraction(closes #28682) 2021-04-07 14:01:48 +01:00
281b8e3443 [jamendo] fix track extraction(closes #28686) 2021-04-07 10:41:06 +01:00
c0c5134c57 [curiositystream] fix format extraction(closes #26845, closes #28668) 2021-04-07 09:27:05 +01:00
72a2c0a9ed release 2021.04.07 2021-04-07 03:42:24 +07:00
445db582a2 [ChangeLog] Actualize
[ci skip]
2021-04-07 03:39:02 +07:00
6b116f0c03 [youtube] Fix videos with restricted location (closes #28685) 2021-04-07 03:34:43 +07:00
70d0d4f9be [compat] Use more conventional name for compat SimpleCookie 2021-04-06 14:22:28 +07:00
6b315d96bc [compat] flake8 2021-04-06 14:15:13 +07:00
25b1287323 [line] add support live.line.me (closes #17205)(closes #28658) 2021-04-05 10:11:01 +01:00
760c911299 [compat] add compat_SimpleCookie to __all__ array 2021-04-05 07:16:50 +01:00
162bf9e10a [compat] add compat_SimpleCookie 2021-04-04 19:49:24 +01:00
6beb1ac65b [extractor/common] keep support for non standard JSON-LD VideoObject author values 2021-04-04 19:16:17 +01:00
3ae9c0f410 [vimeo] improve extraction(closes #28591) 2021-04-04 16:28:26 +01:00
e165f5641f [extractor/common] fix JSON-LD VideoObject author extraction 2021-04-04 16:28:26 +01:00
aee6feb02a [youku] Update ccode(closes #17852, closes #28447, closes #28460) (#28648) 2021-04-04 08:14:37 +00:00
654b4f4ff2 [youtube] prioritize information from YoutubeIE for playlist entries(closes #28619, closes #28636) 2021-04-03 08:23:35 +01:00
1df2596f81 [extractor/common] fix _get_cookies method for python 2(#20673, #23256, #20326, closes #28640) 2021-04-03 07:54:16 +01:00
04d4a3b136 [screencastomatic] fix extraction(closes #11976, closes #24489) 2021-04-01 19:05:00 +01:00
392c467f95 [palcomp3] Add new extractor(closes #13120) 2021-04-01 17:10:38 +01:00
Vid
c5aa8f36bf [arnes] Add new extractor(closes #28483) 2021-04-01 13:59:12 +01:00
3748863070 [youtube:tab] Add support for hashtag videos extraction(closes #28308) 2021-04-01 11:52:23 +01:00
ca304beb15 release 2021.04.01 2021-04-01 04:47:11 +07:00
e789bb1aa4 [ChangeLog] Actualize
[ci skip]
2021-04-01 04:43:54 +07:00
14f29f087e [youtube] Setup CONSENT cookie when needed (closes #28604) 2021-04-01 04:05:10 +07:00
b97fb2edac [vimeo] fix password protected review extraction(closes #27591) 2021-03-31 20:07:13 +01:00
28bab774a0 [youtube] imporve age-restricted video extraction(#28578) 2021-03-30 21:45:08 +01:00
8f493de9fb release 2021.03.31 2021-03-31 02:59:07 +07:00
207bc35d34 [ChangeLog] Actualize
[ci skip]
2021-03-31 02:58:01 +07:00
955894e72f [vlive] fix inkey request(closes #28589) 2021-03-30 10:01:06 +01:00
287e50b56b [francetvinfo] Improve video id extraction (closes #28584) 2021-03-30 03:37:43 +07:00
da762c4e32 [instagram] Improve title extraction and extract duration (#28469)
Co-authored-by: Sergey M. <dstftw@gmail.com>
2021-03-30 02:05:19 +07:00
87a8bde777 [sbs] add support for ondemand watch URLs(closes #28566) 2021-03-28 08:46:33 +01:00
49fc0a567f [youtube] fix video's channel extraction(closes #28562) 2021-03-27 19:11:41 +01:00
cc777dcaa0 [picarto] fix live stream extraction(closes #28532) 2021-03-27 17:37:45 +01:00
c785911870 [vimeo] fix unlisted video extraction(closes #28414) 2021-03-25 17:06:57 +01:00
605e7b5e47 [youtube:tab] fix playlist/comunity continuation items extraction(closes #28266) 2021-03-25 12:53:18 +01:00
8562218350 [ard] improve clip id extraction(#22724)(closes #28528) 2021-03-24 19:29:25 +01:00
76da1c954a release 2021.03.25 2021-03-25 00:04:10 +07:00
c2fbfb49da [ChangeLog] Actualize
[ci skip]
2021-03-25 00:03:00 +07:00
d1069d33b4 [zoom] Add new extractor(closes #16597, closes #27002, closes #28531) 2021-03-24 17:26:44 +01:00
eafcadea26 [extractor] escape forgotten dot for hostnames in regular expression (#28530) 2021-03-24 14:33:19 +00:00
a40002444e [bbc] fix BBC IPlayer Episodes/Group extraction(closes #28360) 2021-03-24 15:10:19 +01:00
5208ae92fc [youtube] Fix default value for youtube_include_dash_manifest (closes #28523) 2021-03-24 02:57:35 +07:00
8117d613ac [zingmp3] fix extraction(closes #11589, closes #16409, closes #16968, closes #27205) 2021-03-22 15:58:56 +01:00
00b4d72d1e [vgtv] Add support for new tv.aftonbladet.se URL schema (#28514)
Co-authored-by: Sergey M <dstftw@gmail.com>
2021-03-22 20:56:58 +07:00
21ccd0d7f4 [tiktok] detect private videos(closes #28453) 2021-03-21 09:10:38 +01:00
7e79ba7dd6 [vimeo:album] Fix extraction for albums with number of videos multiple to page size (closes #28486) 2021-03-20 05:47:26 +07:00
fa6bf0a711 [vvvvid] fix kenc format extraction(closes #28473) 2021-03-19 12:37:22 +01:00
f912d6c8cf [mlb] fix video extracion(#21241) 2021-03-15 21:46:39 +01:00
357bfe251d [svtplay] Improve extraction (closes #28448) 2021-03-15 20:42:20 +07:00
3be098010f [applepodcasts] fix extraction(closes #28445) 2021-03-14 20:08:46 +01:00
9955bb4a27 [rtve] improve extraction
- extract all formats
- fix RTVE Infantil extraction(closes #24851)
- extract is_live and series
2021-03-14 15:05:25 +01:00
ebfd66c4b1 release 2021.03.14 2021-03-14 09:38:16 +07:00
b509d24b2f [ChangeLog] Actualize
[ci skip]
2021-03-14 09:36:11 +07:00
1860d0f41c [southpark] Fix extraction and add support for southparkstudios.com (closes #26763, closes #28413) 2021-03-14 09:26:54 +07:00
60845121ca [sportdeutschland] fix extraction(closes #21856)(closes #28425) 2021-03-13 15:19:24 +01:00
1182f9567b [pinterest] reduce the number of HLS format requests 2021-03-12 18:11:11 +01:00
ef414343e5 [peertube] improve thumbnail extraction(closes #28419) 2021-03-12 10:48:58 +01:00
43d986acd8 [tver] improve title extraction(closes #28418) 2021-03-12 10:14:28 +01:00
9c644a6419 [fujitv] fix HLS formats extension(closes #28416) 2021-03-12 09:51:01 +01:00
fc2c6d5323 [shahid] fix format extraction(closes #28383) 2021-03-10 13:16:21 +01:00
64ed3af328 [lbry] add support for channel filters(closes #28385) 2021-03-10 11:45:30 +01:00
bae7dbf78b [bandcamp] Extract release_timestamp 2021-03-10 03:41:21 +07:00
15c24b0346 [lbry] Extract release_timestamp (closes #28386) 2021-03-10 03:40:56 +07:00
477bff6906 Introduce release_timestamp meta field (refs #28386) 2021-03-10 03:36:31 +07:00
1a1ccd9a6e [pornhub] Detect flagged videos 2021-03-10 02:56:01 +07:00
7dc513487f [pornhub] Extract formats from get_media end point (#28395) 2021-03-10 02:54:10 +07:00
c6a14755bb [bilibili] fix video info extraction(closes #28341) 2021-03-08 16:53:50 +01:00
7f064d50db [cbs] add support for Paramount+ (closes #28342) 2021-03-07 08:32:37 +01:00
b8b622fbeb [trovo] Add Origin header to VOD formats(closes #28346) 2021-03-04 17:57:16 +01:00
ec64ec9651 [voxmedia] fix volume embed extraction(closes #28338) 2021-03-04 12:42:31 +01:00
f68692b004 release 2021.03.03 2021-03-03 11:47:34 +07:00
8c9766f4bf [ChangeLog] Actualize
[ci skip]
2021-03-03 11:44:49 +07:00
061c030133 [youtube:tab] Switch continuation to browse API (closes #28289, closes #28327)
Until further investigation.
2021-03-03 11:42:59 +07:00
8f56907afa [9c9media] fix extraction for videos with multiple ContentPackages(closes #28309) 2021-03-02 12:04:31 +01:00
e1adb3ed4f [bbc] correct catched exception type 2021-03-02 11:21:49 +01:00
e465b25c1f [bbc] add support for BBC Reel videos(closes #21870, closes #23660, closes #28268) 2021-03-02 10:49:20 +01:00
7c06216abf release 2021.03.02 2021-03-02 06:19:42 +07:00
0002888627 [ChangeLog] Actualize
[ci skip]
2021-03-02 06:16:41 +07:00
3fb14cd214 [zdf] Rework extractors (closes #11606, closes #13473, closes #17354, closes #21185, closes #26711, closes #27068, closes #27930, closes #28198, closes #28199, closes #28274)
* Generalize unique video ids for zdf based extractors
* Improve extraction
* Fix 3sat and phoenix
2021-03-02 06:07:30 +07:00
bee6182680 [stretchinternet] Fix extraction(closes #28297) 2021-03-01 14:00:03 +01:00
38fe5e239a [urplay] fix episode data extraction(closes #28292) 2021-02-28 12:31:18 +01:00
678d46f6bb [bandaichannel] Add new extractor(closes #21404) 2021-02-28 10:42:41 +01:00
3c58f9e0b9 [srgssr] improve extraction
- extract subtitle
- fix extraction for new videos
- update srf download domains

closes #14717
closes #14725
closes #27231
closes #28238
2021-02-25 15:50:49 +01:00
ef28e33249 [vvvvid] reduce season request payload size 2021-02-24 22:29:35 +01:00
9662e4964b [vvvvid] extract series sublists playlist_title (#27601) (#27618) 2021-02-24 22:17:29 +01:00
44603290e5 [dplay] Extract Ad-Free uplynk URLs(#28160) 2021-02-24 18:34:28 +01:00
1631fca1ee [wat] detect DRM protected videos(closes #27958) 2021-02-23 13:50:18 +01:00
295860ff00 [tf1] improve extraction(closes #27980)(closes #28040) 2021-02-23 12:41:32 +01:00
8cb4b71909 [tmz] Fix and improve extraction (closes #24603, closes #24687, closes #28211) 2021-02-23 18:37:06 +07:00
d81421af4b [gedidigital] improve asset id matching 2021-02-22 23:02:15 +01:00
7422a2194f [gedidigital] Add new extractor(closes #7347)(closes #26946) 2021-02-22 20:42:14 +01:00
2090dbdc8c [youtube] fix get_video_info request 2021-02-21 23:09:09 +01:00
0a04e03a02 release 2021.02.22 2021-02-22 02:42:16 +07:00
44b2d5f5fc [ChangeLog] Actualize
[ci skip]
2021-02-22 02:40:00 +07:00
aa9118a373 [apa] Improve extraction (closes #27750) 2021-02-22 02:29:50 +07:00
36abc16c3c [apa] Fix extraction 2021-02-22 02:28:28 +07:00
919d764600 [youporn] Skip test 2021-02-21 23:21:38 +07:00
696183e133 [youporn] Extract duration (#28019)
Co-authored-by: Sergey M <dstftw@gmail.com>
2021-02-21 23:19:37 +07:00
f90d825a6b [peertube] Add support for canard.tube (#28190) 2021-02-21 23:05:33 +07:00
3037ab00c7 [youtube] fixup m4a_dash formats(closes #28165) 2021-02-21 10:31:27 +01:00
21e872b19a [samplefocus] Add new extractor(closes #27763) 2021-02-20 10:55:19 +01:00
cf2dbec630 [vimeo] add support for unlisted video source format extraction 2021-02-19 21:13:56 +01:00
b92bb0e02a [viki] improve extraction(closes #26522)(closes #28203)
- extract uploader_url and episode_number
- report login required error
- extract 480p formats
- fix API v4 calls
2021-02-19 16:00:22 +01:00
40edffae3d [ninegag] unscape title(#28201) 2021-02-19 11:55:40 +01:00
9fc5eafb8e [youtube] Improve _VALID_URL (refs #28193) 2021-02-18 04:59:56 +07:00
08c2fbb844 [youtube] Add support for redirect.invidious.io (#28193)
Co-authored-by: Sergey M <dstftw@gmail.com>
2021-02-18 04:29:32 +07:00
3997efb65e [dplay] add support for de.hgtv.com (closes #28182) 2021-02-17 19:50:04 +01:00
a7356dffe9 [dplay] Add support for discoveryplus.com (closes #24698) 2021-02-17 18:33:33 +01:00
e20ec43094 [simplecast] Add new extractor(closes #24107) 2021-02-17 14:53:23 +01:00
70baa7bfae [test_youtube_lists] Actualize youtube flat playlist test (closes #28045) 2021-02-17 04:58:54 +07:00
8980f53b42 [youtube] Fix uploader extraction in flat playlist mode (#28045) 2021-02-17 04:21:33 +07:00
a363fb5d28 [yandexmusic:playlist] Request missing tracks in chunks (closes #27355, closes #28184) 2021-02-17 04:03:54 +07:00
Max
646052e416 [postprocessor/embedthumbnail] Recognize atomicparsley binary in lowercase (#28112) 2021-02-17 03:22:51 +07:00
844e4cbc54 [storyfire] Add new extractor(closes #25628)(closes #26349) 2021-02-16 21:14:43 +01:00
56c63c8c02 [zhihu] Add new extractor(closes #28177) 2021-02-16 10:08:43 +01:00
07eb8f1916 [youtube] Fix controversial videos when authenticated with cookies (closes #28174) 2021-02-16 05:57:53 +07:00
4b5410c5c8 [ccma] fix timestamp parsing in python 2 2021-02-15 13:06:54 +01:00
be2e9b76ee [videopress] add support for video.wordpress.com 2021-02-14 22:10:06 +01:00
d8085580f6 [kakao] improve info extraction and detect geo restriction(closes #26577) 2021-02-14 19:48:26 +01:00
6d32c6c6d3 [xboxclips] fix extraction(closes #27151) 2021-02-14 16:22:45 +01:00
f94d764993 [ard] Improve formats extraction (closes #28155) 2021-02-14 05:03:15 +07:00
f28f1b4d6e [canvas] Add new extractor for Dagelijkse Kost (#28119) 2021-02-11 08:04:16 +00:00
360d5f0daa release 2021.02.10 2021-02-10 22:34:47 +07:00
cd493c5adc [ChangeLog] Actualize
[ci skip]
2021-02-10 22:32:25 +07:00
a4c7ed6b1e [youtube:tab] Improve grid continuation extraction (closes #28130) 2021-02-10 22:28:58 +07:00
7f8b8bc418 [ign] fix extraction(closes #24771) 2021-02-08 15:58:20 +01:00
311ebdd9a5 [xhamster] Extract formats from xplayer settings and extract filesizes (closes #28114) 2021-02-08 15:47:12 +07:00
99c68db0a8 [youtube] add support phone/tablet JS player(closes #26424) 2021-02-08 09:20:28 +01:00
5fc53690cb [archiveorg] Fix and improve extraction (closes #21330, closes #23586, closes #25277, closes #26780, closes #27109, closes #27236, closes #28063) 2021-02-07 20:34:41 +07:00
7a9161578e [cda] Detect geo restricted videos (refs #28106) 2021-02-07 19:18:40 +07:00
2405854705 [urplay] Fix extraction (closes #28073) (#28074) 2021-02-07 02:46:05 +07:00
0cf09c2b41 [youtube] Fix release date extraction (closes #28094) 2021-02-07 02:17:03 +07:00
0156ce95c5 [youtube] Extract abr and vbr (closes #28100) 2021-02-07 02:03:47 +07:00
1641b13232 [youtube] skip OTF formats(#28070) 2021-02-04 13:05:35 +01:00
a4bdc3112b release 2021.02.04.1 2021-02-04 13:11:33 +07:00
c7d407bca2 [ChangeLog] Actualize
[ci skip]
2021-02-04 13:09:28 +07:00
7215691ab7 [youtube] Prefer DASH formats (closes #28070) 2021-02-04 13:07:43 +07:00
fc88e8f0e3 [azmedien] Fix extraction (#28064) 2021-02-03 23:57:56 +00:00
cfefb7d854 release 2021.02.04 2021-02-04 04:49:25 +07:00
3c07d007ca [ChangeLog] Actualize
[ci skip]
2021-02-04 04:47:30 +07:00
89c5a7d5aa [pornhub] Implement lazy playlist extraction 2021-02-04 04:42:14 +07:00
2adc0c51cd [pornhub] Add placeholder netrc machine 2021-02-04 04:20:09 +07:00
1f0910bc27 [svtplay] Fix video id extraction (closes #28058) 2021-02-04 04:17:45 +07:00
e22ff4e356 [pornhub] Add support for authentication (closes #18797, closes #21416, closes #24294) 2021-02-04 04:09:11 +07:00
83031d749b [pornhub:user] Add support for URLs unavailable via /videos page and improve paging (closes #27853) 2021-02-04 00:25:53 +07:00
1b731ebcaa [bravotv] add support for oxygen.com(closes #13357)(closes #22500) 2021-02-03 18:13:17 +01:00
ab25f3f431 [youtube] pass embed URL to get_video_info request 2021-02-03 17:15:31 +01:00
07f7aad81c [ccma] improve metadata extraction(closes #27994)
- extract age_limit, alt_title, categories, series and episode_number
- fix timestamp multiple subtitles extraction
2021-02-03 09:19:54 +01:00
1e2575df87 Credit @adrianheine for #27732 2021-02-03 00:21:46 +01:00
b111a64135 [egghead] fix typo 2021-02-02 19:05:37 +01:00
0e3a968479 [egghead] update API domain(closes #28038) 2021-02-02 19:00:36 +01:00
c11f7cf9bd [vidzi] remove extractor(closes #12629) 2021-02-01 22:35:28 +01:00
8fa7cc387d [vidio] improve metadata extraction 2021-02-01 21:35:18 +01:00
65eee5a745 [youtube] improve subtitle extraction 2021-02-01 18:12:35 +01:00
efef4ddf51 [youtube] fix chapter extraction fallback 2021-02-01 16:49:52 +01:00
159a3d48df [youtube] keep _formats array for format sorting tests 2021-02-01 16:36:19 +01:00
b46483a6ec [youtube/test_youtube_signature] fix test 2021-02-01 16:35:07 +01:00
9c724601ba [youtube] remove description chapters tests
video description no longer contain yt.www.watch.player.seekTo
function
2021-02-01 16:11:07 +01:00
67299f23d8 [youtube] Rewrite Extractor
- improve format sorting
- remove unused code(swf parsing, ...)
- fix series metadata extraction
- fix trailer video extraction
- improve error reporting
- extract video location
2021-02-01 14:53:01 +01:00
8bf9591a70 [AENetworks] update AENetworksShowIE test playlist id (#27851) 2021-01-30 13:44:01 +01:00
a800838f5a [vvvvid] add support for youtube embeds (#27825) 2021-01-29 14:24:28 +01:00
ba15b2fee6 [googledrive] report download page errors(closes #28005) 2021-01-29 11:53:49 +01:00
56a7ee9033 [vlive] Fix error message decoding for python 2 (#28004) 2021-01-29 08:02:18 +00:00
0b4f03a563 [youtube] improve DASH formats file size extraction 2021-01-28 16:24:46 +01:00
7b8fa658f8 [cda] Improve birth validation detection (closes #14022) (#27929)
Co-authored-by: Sergey M <dstftw@gmail.com>
2021-01-28 02:43:20 +07:00
fd95fc33b1 [awaan] Extract uploader id (#27963) 2021-01-28 02:06:12 +07:00
c669554ef5 [medialaan] add support DPG Media MyChannels based websites
closes #14871
closes #15597
closes #16106
closes #16489
2021-01-27 14:58:47 +01:00
11b68df7a4 [abcnews] fix extraction(closes #12394)(closes #27920) 2021-01-27 12:28:22 +01:00
d18f4419a7 [AMP] Fix upload_date and timestamp extraction (#27970) 2021-01-26 22:43:11 +01:00
0f7d413d5b [tv4] relax _VALID_URL(closes #27964) 2021-01-25 15:15:45 +01:00
286e5d6724 [tv2] improve MTV Uutiset Article extraction 2021-01-25 14:46:04 +01:00
395981288b [tv2] Add support for mtvuutiset.fi (#27744) 2021-01-25 14:43:41 +01:00
55bb3556c8 [adn] improve login warning reporting 2021-01-25 09:28:24 +01:00
57f2488bbe [zype] fix uplynk id extraction(closes #27956) 2021-01-25 09:26:22 +01:00
ea399a53eb [ADN] Implement login (#27937)
closes #17091
closes #27841
2021-01-25 08:25:57 +00:00
811a183eb6 [ChangeLog] Actualize
[ci skip]
2021-01-24 18:15:32 +07:00
b63981e850 release 2021.01.24.1 2021-01-24 18:11:20 +07:00
186cbaffb9 release 2021.01.24 2021-01-24 18:00:58 +07:00
dbf3fa8af6 [ChangeLog] Actualize
[ci skip]
2021-01-24 17:36:40 +07:00
f08c31cf33 [franceculture] Make thumbnail optional (closes #18807) 2021-01-23 01:10:35 +07:00
d8dab85419 [franceculture] Fix extraction (closes #27891) (#27903)
Co-authored-by: Sergey M. <dstftw@gmail.com>
2021-01-23 01:03:45 +07:00
5519bba3e1 [options] Clarify --extract-audio help string (closes #27878) 2021-01-23 00:00:53 +07:00
142c584063 Introduce --output-na-placeholder (closes #27896) 2021-01-22 23:54:29 +07:00
4542e3e555 [njpwworld] fix extraction (#27890) 2021-01-22 08:56:51 +00:00
fa8f6d8580 [comedycentral] fix extraction(closes #27905) 2021-01-21 23:53:09 +01:00
3bb7769c40 [wat] remove unused variable 2021-01-21 17:22:30 +01:00
8d286bd5b6 [wat] fix format extraction(closes #27901) 2021-01-21 17:20:32 +01:00
cff72b4cc0 [americastestkitchen] improve season extraction 2021-01-21 16:47:55 +01:00
657221c81d [americastestkitchen] Add support for downloading entire seasons (#27861) 2021-01-21 15:46:29 +00:00
62acf5fa2c [trovo] Add new extractor(closes #26125) 2021-01-19 22:43:59 +01:00
b79977fb6b [aol] add support for yahoo videos(closes #26650) 2021-01-19 14:49:45 +01:00
bc7c8f3d4e [yahoo] fix single video extraction 2021-01-19 14:47:39 +01:00
015e19b350 [lbry] unescape lbry uri(closes #27872) 2021-01-19 11:54:39 +01:00
54856480d7 [ninegag] improve extraction 2021-01-19 10:23:02 +01:00
1dd12708c2 [9gag] Fix Extraction (#23022) 2021-01-19 10:21:37 +01:00
f9201cef58 [americastestkitchen] Improve metadata extraction for ATK episodes (#27860) 2021-01-18 14:47:06 +00:00
26499ba823 [aljazeera] fix extraction(closes #20911)(closes #27779) 2021-01-17 17:35:10 +01:00
58f6c2112d [minds] improve extraction 2021-01-17 14:07:56 +01:00
de026a6acd [Minds] Add new extractor (#17934) 2021-01-17 13:05:39 +00:00
d4564afc70 [ard] Fix title and description extraction and update tests (#27761) 2021-01-17 05:41:52 +07:00
360a5e0f60 [aenetworks] Fix test (#27847) 2021-01-17 03:40:08 +07:00
55a3ca16d3 [spotify] Add new extractor for Spotify Podcasts(closes #27443) 2021-01-16 18:12:05 +01:00
ef50cb3fda release 2021.01.16 2021-01-16 09:51:30 +07:00
8673f4344c [ChangeLog] Actualize
[ci skip]
2021-01-16 09:48:00 +07:00
f1487d4fca [mixcloud:playlist:base] Fix video id extraction in flat playlist mode (refs #27787) 2021-01-16 09:42:57 +07:00
0cd4c402f0 [animeondemand] Add support for lazy playlist extraction (closes #27829) 2021-01-16 09:22:33 +07:00
9c9b458145 [YoutubeDL] Protect from infinite recursion due to recursively nested playlists (closes #27833) 2021-01-16 01:46:41 +07:00
9d50f86232 [twitter] Add tests for more cards 2021-01-15 10:32:01 +01:00
7e92f9015e [youporn] Restrict fallback download URL (refs #27822) 2021-01-15 15:12:04 +07:00
aa860b8016 [youporn] Improve height and tbr extraction (refs #23659, refs #20425) 2021-01-15 14:54:23 +07:00
b484097b01 [youporn] Fix extraction (closes #27822) 2021-01-15 14:43:52 +07:00
ab9001dab5 [twitter] Add support for unified cards (closes #27826) 2021-01-15 12:39:21 +07:00
879866a230 [twitch] Set OAuth token for GraphQL requests using auth-token cookie (#27790)
Co-authored-by: remitamine <remitamine@gmail.com>
2021-01-14 12:41:44 +00:00
8e5477d036 [YoutubeDL] Ignore failure to create existing directory (#27811) 2021-01-14 11:46:07 +07:00
1e8e5d5238 [workflows/ci.yml] Run CI on PRs 2021-01-14 02:54:38 +07:00
d81a213cfb [YoutubeDL] Raise syntax error for format selection expressions with multiple + operators (closes #27803) 2021-01-14 00:38:31 +07:00
7c2d18a13f [Mixcloud] Harmonize ID generation from lists with full ID generation (#27787)
Mixcloud IDs are generated as `username_slug` when the full ID dict has been
downloaded.  When downloading a list (e.g. uploads, favorites, ...), the temporary
ID is just the `slug`.  This made e.g. archive file usage require the download
of stream metadata before the download can be rejected as already downloaded.

This commit attempts to get the uploader username during the GraphQL query, so the
temporary IDs are generated similarly.
2021-01-13 09:22:48 +00:00
2408e6d26a [cspan] improve info extraction(closes #27791) 2021-01-13 09:17:43 +01:00
cf862771d7 [adn] improve info extraction 2021-01-12 21:15:13 +01:00
a938f111ed [ADN] Fix extraction (#27732)
Closes #26963.
2021-01-12 20:12:44 +00:00
4759543f6e [youtube:search] fix view_count and try to extract all video sections(closes #27588)(closes #27604) 2021-01-11 17:35:51 +01:00
d0fc289f45 [workflows/ci.yml] Change ubuntu-latest to ubuntu-18.04
ubuntu-latest alias will be switched to ubuntu-20.04 soon (see [1]) which lacks python 3.3 and 3.4 (see [2]).

1. https://github.blog/changelog/2020-10-29-github-actions-ubuntu-latest-workflows-will-use-ubuntu-20-04/
2. https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json
2021-01-11 01:30:54 +07:00
70f572585d [twitch] Improve login error extraction 2021-01-10 06:23:58 +07:00
c2d06aef60 [twitch] Fix authentication (refs #27743) 2021-01-10 06:23:08 +07:00
ff1e765400 [threeqsdn] Improve extraction(closes #21058) 2021-01-09 22:29:58 +01:00
170e1c1995 [peertube] Extract files also from streamingPlaylists (#27728)
JSON objects with an empty "files" tag seem to be a valid PeerTube API
response. In those cases the "files" arrays contained in the
"streamingPlaylists" members can be used instead.
closes #26002
closes #27586
2021-01-08 20:09:38 +00:00
61e669acff [khanacademy] fix extraction(closes #2887)(closes #26803) 2021-01-08 16:13:22 +01:00
2c337f4e85 [spike] Update Paramount Network feed URL(closes #27715) 2021-01-08 09:26:12 +01:00
bf6a74c620 release 2021.01.08 2021-01-08 11:03:34 +07:00
38a967c98e [ChangeLog] Actualize
[ci skip]
2021-01-08 11:01:29 +07:00
3a61e6d360 [rai] improve subtitles extraction (#27705)
closes #27698
2021-01-07 13:48:45 +00:00
3d8e32dcc0 [canvas] Match only supported VRT NU URLs(#27707) 2021-01-07 12:35:04 +01:00
8f29b2dd38 [extractors] add BibelTVIE import 2021-01-06 20:39:04 +01:00
a29e340efa [bibeltv] Add new extractor(closes #14361) 2021-01-06 20:38:02 +01:00
b13f29098f [bfmtv] Add new extractor(closes #16053)(closes #26615) 2021-01-06 18:50:02 +01:00
430c4bc9d0 [sbs] Add support for ondemand play and news embed URLs(closes #17650)(closes #27629) 2021-01-06 13:41:07 +01:00
4ae243fc6c [twitch] Refactor 2021-01-06 03:54:33 +07:00
8f20ad36dc [twitch] Drop legacy kraken API v5 code altogether 2021-01-06 03:46:45 +07:00
799c794947 [twitch:vod] Switch to GraphQL for video metadata 2021-01-06 03:34:36 +07:00
1ae7ae0b96 [canvas] Fix VRT NU extraction(closes #26957)(closes #27053) 2021-01-05 21:17:39 +01:00
ccc7112291 [twitch] Improve access token extraction and remove unused code (closes #27646) 2021-01-06 02:11:49 +07:00
5b24f8f505 [twitch] Switch access token to GraphQL and refactor. 2021-01-06 02:11:42 +07:00
fcd90d2583 [rai] Detect ContentItem in iframe (closes #12652) (#27673)
Co-authored-by: Sergey M. <dstftw@gmail.com>
2021-01-06 01:49:56 +07:00
8f757c7353 [ketnet] fix extraction(closes #27662) 2021-01-05 18:17:04 +01:00
be1a3f2d11 [dplay] Add suport Discovery+ domains(closes #27680) 2021-01-05 17:31:58 +01:00
ecae54a98d [motherless] Fix review issues and improve extraction (closes #26495, closes #27450) 2021-01-05 07:40:30 +07:00
f318882955 [motherless] Fix recent videos upload date extraction (closes #27661)
Less than a week old videos use a '20h ago' or '1d ago' format.

I kept the support for 'Ago' with uppercase start at is was already in the code.
2021-01-05 07:40:25 +07:00
c3399cac19 [travis] Drop Travis CI (closes #7193, closes #12366) 2021-01-05 02:44:27 +07:00
9237aaa77f [workflows/ci.yml] Add support for jython 2021-01-05 02:38:44 +07:00
766fcdd0fa [downloader/hls] Disable decryption in tests (#27660)
Tests truncate the download to 10241 bytes, which is not divisible by 16
and cannot be decrypted. Tests don't really care about the decrypted
content, just that the data they retrieved is the expected data.
Therefore, it's fine to just return the encrypted data to tests.

See: #27621 and #27620
2021-01-05 01:24:01 +07:00
f6ea29e24b [iheart] Update test description value (#27037)
the description has no HTML tags now.
2021-01-04 17:16:27 +01:00
8a3797a4ab [nrk] fix extraction for videos without a legalAge rating 2021-01-04 12:16:54 +01:00
745db8899d [iheart] clean HTML tags from episode description 2021-01-04 09:59:20 +01:00
83db801cbf [iheart] remove print statement 2021-01-04 09:42:27 +01:00
964a8eb754 [googleplus] Remove Extractor(closes #4955)(closes #7400) 2021-01-04 01:14:26 +01:00
ac61f2e058 [applepodcasts] Add new extractor(#25918) 2021-01-04 01:14:26 +01:00
8487e8b98a [googlepodcasts] Add new extractor 2021-01-04 01:14:26 +01:00
9c484c0019 [iheart] Add new extractor for iHeartRadio(#27037) 2021-01-04 01:14:26 +01:00
0e96b4b5ce [acast] clean podcast URLs 2021-01-04 01:14:26 +01:00
a563c97c5c [stitcher] clean podcast URLs 2021-01-04 01:14:25 +01:00
e88c9ef62a [utils] add a function to clean podcast URLs 2021-01-04 01:14:25 +01:00
0889eb33e0 [xfileshare] Add support for aparat.cam (closes #27651) 2021-01-04 00:17:51 +07:00
0021a2b9a1 [nrktv] Add subtitles test 2021-01-03 22:25:17 +07:00
19ec468635 [twitter] Add support for summary card(closes #25121) 2021-01-03 13:37:24 +01:00
491ee7efe4 [twitter] try to use a Generic fallback for unknown twitter cards(closes #25982) 2021-01-03 13:29:00 +01:00
8522bcd97c [stitcher] Add support for shows and show metadata extraction(closes #20510) 2021-01-03 12:12:06 +01:00
ac71fd5919 [stv] improve episode id extraction(closes #23083) 2021-01-03 10:04:57 +01:00
8e953dcbb1 release 2021.01.03 2021-01-03 09:12:47 +07:00
f4afb9a6a8 [ChangeLog] Actualize
[ci skip]
2021-01-03 09:10:40 +07:00
d5b8cf093c [nrk] Fix age limit extraction 2021-01-03 08:58:05 +07:00
5c6e84c0ff [nrk] Improve series metadata extraction (closes #27473) 2021-01-03 08:31:26 +07:00
1aaee908b9 [nrk] PEP 8 2021-01-03 07:57:12 +07:00
b2d9fd9c9f [nrk] Improve episode and season number extraction 2021-01-03 07:53:05 +07:00
bc2f83b95e [nrktv] Fix tests 2021-01-03 07:31:03 +07:00
85de33b04e [nrk] Improve series metadata extraction 2021-01-03 07:30:45 +07:00
7dfd966848 [nrk] Extract subtitles 2021-01-03 07:02:13 +07:00
a25d03d7cb [nrk] Fix age limit extraction 2021-01-03 06:51:21 +07:00
cabfd4b1f0 [nrk] Inline _extract_from_playback 2021-01-03 06:45:17 +07:00
7b643d4cd0 [nrk] Improve video id extraction 2021-01-03 06:44:28 +07:00
1f1d01d498 [nrk] Add more shortcut tests 2021-01-03 06:41:37 +07:00
21a42e2588 [nrk] Improve extraction (closes #27634, closes #27635)
+ Add support for mp3 formats
* Generalize and delegate all item extractors to nrk, beware ie key breakages
+ Add support for podcasts
+ Generalize nrk shortcut form to support all kind of ids
2021-01-03 06:29:52 +07:00
2df93a0c4a [nrktv] Switch to playback endpoint
mediaelement endpoint is no longer in use.
2021-01-03 05:01:45 +07:00
75972e200d [vvvvid] fix season metadata extraction(#18130) 2021-01-01 20:21:59 +01:00
d0d838638c [stitcher] fix extraction(closes #20811)(closes #27606) 2021-01-01 18:38:46 +01:00
8c17afc471 [acast] fix extraction(closes #21444)(closes #27612)(closes #27613) 2021-01-01 17:05:42 +01:00
40d66e07df [arcpublishing] add missing staticmethod decorator 2020-12-31 23:42:17 +01:00
ab89a8678b [arcpublishing] Add new extractor
closes #2298
closes #9340
closes #17200
2020-12-31 23:31:07 +01:00
4d7d056909 [sky] add support for Sports News articles and Brighcove videos(closes #13054) 2020-12-31 12:18:30 +01:00
c35bc82606 [vvvvid] skip unplayable episodes and extract akamai formats(closes #27599) 2020-12-31 12:18:30 +01:00
2f56caf083 [yandexvideo] fix extraction for Python 3.4 2020-12-31 12:18:30 +01:00
4066945919 release 2020.12.31 2020-12-31 05:17:55 +07:00
2a84694b1e [ChangeLog] Actualize
[ci skip]
2020-12-31 05:14:33 +07:00
4046ffe1e1 [redditr] Fix review issues and extract source thumbnail (closes #27503) 2020-12-31 05:07:57 +07:00
d1d0612160 [redditr] Extract all thumbnails 2020-12-31 05:07:51 +07:00
7b0f04ed1f [vvvvid] imporove info extraction 2020-12-30 18:16:47 +01:00
2e21b06ea2 [vvvvid] add playlists support (#27574)
closes #18130
2020-12-30 18:12:17 +01:00
a6f75e6e89 [yandexdisk] extract info from webpage
the public API does not return metadata when download limit is reached
2020-12-30 16:45:53 +01:00
bd18824c2a [yandexdisk] fix extraction(closes #17861)(closes #27131) 2020-12-30 13:43:56 +01:00
bdd044e67b [yandexvideo] use old api call as fallback 2020-12-30 13:30:11 +01:00
f7e95fb2a0 [yandexvideo] fix extraction(closes #25000) 2020-12-30 09:30:30 +01:00
9dd674e1d2 [utils] accept only supported protocols in url_or_none 2020-12-30 09:22:30 +01:00
9c1e164e0c [YoutubeDL] Allow format filtering using audio language(#16209) 2020-12-29 19:29:08 +01:00
c706fbe9fe [nbc] Remove CSNNE extractor 2020-12-29 17:21:05 +01:00
ebdcf70b0d [nbc] fix NBCSport VPlayer URL extraction(closes #16640) 2020-12-29 17:15:13 +01:00
5966095e65 [aenetworks] fix HistoryPlayerIE tests 2020-12-29 16:59:31 +01:00
9ee984fc76 [aenetworks] add support for biography.com (closes #3863) 2020-12-29 16:13:36 +01:00
53528e1d23 [uktvplay] match new video URLs(closes #17909) 2020-12-29 14:11:37 +01:00
c931c4b8dd [sevenplay] detect API errors 2020-12-29 14:11:37 +01:00
7acd042bbb [tenplay] fix format extraction(closes #26653) 2020-12-29 14:11:37 +01:00
bcfe485e01 [brightcove] raise ExtractorError for DRM protected videos(closes #23467)(closes #27568) 2020-12-29 14:11:37 +01:00
479cc6d5a1 release 2020.12.29 2020-12-29 02:52:31 +07:00
38286ee729 [ChangeLog] Actualize
[ci skip]
2020-12-29 02:49:53 +07:00
1a95953867 [youtube] Improve yt initial data extraction (closes #27524) 2020-12-29 02:29:34 +07:00
71febd1c52 [youtube:tab] Improve URL matching (closes #27559) 2020-12-29 02:19:43 +07:00
f1bc56c99b [youtube:tab] Restore retry on browse requests (closes #27313, closes #27564) 2020-12-29 02:11:48 +07:00
64e419bd73 [aparat] Fix extraction
closes #22285
closes #22611
closes #23348
closes #24354
closes #24591
closes #24904
closes #25418
closes #26070
closes #26350
closes #26738
closes #27563
2020-12-28 18:19:30 +01:00
782ea947b4 [brightcove] remove sonyliv specific code 2020-12-28 11:12:57 +01:00
f27224d57b [piksel] import format extraction 2020-12-28 10:50:29 +01:00
c007188598 [zype] Add support for uplynk videos 2020-12-27 23:47:28 +01:00
af93ecfd88 [toggle] add support for live.mewatch.sg (closes #27555) 2020-12-27 22:26:20 +01:00
794771a164 [go] Added support for FXNetworks (#26826)
Co-authored-by: James Kirrage <james.kirrage@mortgagegym.com>

closes #13972
closes #22467
closes #23754
2020-12-27 17:36:21 +00:00
6f2eaaf73d [teachable] Improve embed detection (closes #26923) 2020-12-27 22:57:50 +07:00
4c7a4dbc4d [mitele] fix free video extraction(#24624)(closes #25827)(closes #26757) 2020-12-27 16:22:43 +01:00
f86b299d0e [telecinco] fix extraction 2020-12-27 16:22:43 +01:00
e474996541 [youtube] Update invidious.snopyta.org (#22667)
Co-authored-by: sofutru <54445344+sofutru@users.noreply.github.com>
2020-12-27 21:15:09 +07:00
aed617e311 [amcnetworks] improve auth only video detection(closes #27548) 2020-12-27 09:00:08 +01:00
0fa67c1d68 [generic] Add support for VHX Embeds(#27546) 2020-12-27 09:00:07 +01:00
365b3cc72d release 2020.12.26 2020-12-26 23:17:35 +07:00
a272fe21a8 [ChangeLog] Actualize
[ci skip]
2020-12-26 23:13:26 +07:00
cec1c2f211 [instagram] Fix test 2020-12-26 23:00:15 +07:00
12053450dc [instagram] Fix comment count extraction 2020-12-26 23:00:15 +07:00
46cffb0c47 [instagram] Add support for reel URLs (closes #26234, closes #26250) 2020-12-26 23:00:15 +07:00
c32a059f52 [bbc] switch to media selector v6
closes #23232
closes #23933
closes #26303
closes #26432
closes #26821
closes #27538
2020-12-26 16:57:02 +01:00
6911312e53 [instagram] Improve thumbnail extraction 2020-12-26 22:42:58 +07:00
f22b5a6b96 [instagram] Improve extraction (closes #22880) 2020-12-26 22:37:41 +07:00
58e55198c1 [instagram] Fix extraction when authenticated (closes #27422) 2020-12-26 22:31:55 +07:00
d61ed9f2f1 [spankbang] Remove unused import 2020-12-26 22:14:31 +07:00
8bc4c6350e [spangbang:playlist] Fix extraction (closes #24087) 2020-12-26 21:58:26 +07:00
cfa4ffa23b [spangbang] Add support for playlist videos 2020-12-26 21:55:12 +07:00
4f1dc1463d [pornhub] Improve like and dislike count extraction (closes #27356) 2020-12-26 21:24:43 +07:00
17e0f41d34 [pornhub] Fix review issues (closes #27393) 2020-12-26 21:17:17 +07:00
b57b27ff8f [pornhub] Fix lq formats extraction (closes #27386) 2020-12-26 21:17:11 +07:00
bbe8cc6662 [README.md] Update reference to cookie export extension for Chrome (closes #26885) (#27433)
The cookies.txt extension doesn't exist anymore on the Chrome Web Store (see https://chrome.google.com/webstore/detail/cookiestxt/njabckikapfpffapmjgojcnbfjonfjfg), so I propose to change the link in the README.md to another similar extension called Get cookies.txt (https://chrome.google.com/webstore/detail/get-cookiestxt/bgaddhkoddajcdgocldbbfleckgcbcid/) with the same functions and utility of the old one.

This PR close #26885
2020-12-26 20:50:39 +07:00
98106accb6 [bongacams] Add extractor (closes #27440) 2020-12-26 20:30:19 +07:00
af1312bfc3 [youtube:tab] Extend _VALID_URL (closes #27501) 2020-12-26 19:59:57 +07:00
4c7d7215cd [theweatherchannel] fix extraction (closes #25930)(closes #26051) 2020-12-26 09:37:40 +01:00
0370d9eb3d [sprout] correct typo 2020-12-25 21:35:55 +01:00
1434651d20 [sprout] Add support for Universal Kids (closes #22518) 2020-12-25 21:17:44 +01:00
2c312ab84a [theplatform] allow passing geo bypass countries from other extractors 2020-12-25 21:16:36 +01:00
0ee78d62d5 [wistia] Add support for playlists (closes #27533) 2020-12-25 17:00:30 +01:00
7f3c90ab25 [ctv] Add new extractor (closes #27525) 2020-12-24 19:40:08 +01:00
1d3cd29730 [9c9media] improve info extraction 2020-12-24 17:39:13 +01:00
4ef1fc9707 [youtube] fix automatic captions extraction(closes #27162)(closes #27388) 2020-12-24 16:05:03 +01:00
f9e6aa1dcf [sonyliv] fix title for movies 2020-12-24 13:33:12 +01:00
f83db9064b [sonyliv] fix extraction(closes #25667) 2020-12-24 13:10:20 +01:00
2da9a86399 [streetvoice] fix extraction(closes #27455)(closes #27492) 2020-12-24 13:10:20 +01:00
ecaa535cf4 [facebook] add support for watchparty pages(closes #27507) 2020-12-24 13:10:20 +01:00
79dd92b1fe [cbslocal] fix video extraction 2020-12-24 13:10:20 +01:00
bd3844c9c2 [brightcove] add another method to extract policyKey 2020-12-24 13:10:20 +01:00
7bf5e3a84a [mewatch] Relax _VALID_URL (closes #27506) 2020-12-22 23:27:05 +07:00
08a17dae5b release 2020.12.22 2020-12-22 04:48:07 +07:00
924ea66ade [ChangeLog] Actualize
[ci skip]
2020-12-22 04:45:37 +07:00
5b72f5b74f [anvato] remove NFLTokenGenerator
until a better solution is introduced that:
- works with lazy_extractors
- allows for 3rd party token generators
2020-12-21 09:02:45 +01:00
bfa345744d [tastytrade] Remove Extractor(closes #25716)
covered by GenericIE via BrighcoveNewIE
2020-12-20 18:07:34 +01:00
f966461476 [niconico] fix playlist extraction(closes #27428) 2020-12-20 17:15:43 +01:00
b8aea53682 [everyonesmixtape] Remove Extractor 2020-12-20 17:10:40 +01:00
c0d9eb7043 [kanalplay] Remove Extractor 2020-12-20 12:06:17 +01:00
3ba6aabd25 [arkena] fix extraction 2020-12-20 12:06:17 +01:00
a8b31505ed Switch to GitHub actions for CI
Travis CI has ignored our requests and does not look to be interested in providing OSS credits for youtube-dl
2020-12-20 06:48:20 +07:00
90a271e914 [nba] rewrite extractor 2020-12-19 20:14:44 +01:00
172d270607 [turner] improve info extraction 2020-12-19 20:14:44 +01:00
22feed08a1 [common] remove unwanted query params from unsigned akamai manifest URLs 2020-12-19 20:14:44 +01:00
942b8ca3be [youtube] Improve xsrf token extraction (closes #27442) 2020-12-20 00:48:44 +07:00
3729c52f9d [generic] Improve RSS age limit extraction 2020-12-19 23:24:52 +07:00
71679eaee8 [generic] Fix RSS itunes thumbnail extraction (#27405) 2020-12-19 23:18:51 +07:00
76fe4ba3b2 [redditr] Extract duration (#27426) 2020-12-17 23:21:36 +07:00
164a4a5756 [anvato] Disable NFLTokenGenerator(closes #27449) 2020-12-16 21:47:50 +01:00
455951985b [zaq1] Remove extractor 2020-12-16 21:20:57 +01:00
c29500e412 [asiancrush] fix extraction and add support for retrocrush.tv
closes #25577
closes #25829
2020-12-16 12:30:41 +01:00
1bc1520adc [noco] Remove Extractor(closes #10864) 2020-12-15 19:07:14 +01:00
022e05dc1f [nfl] fix extraction(closes #22245) 2020-12-15 16:06:10 +01:00
b34c9551aa [anvato] update ANVACK table and add experimental token generator for NFL 2020-12-15 16:01:33 +01:00
84f19c026f [sky] relax SkySports URL regex (closes #27435) 2020-12-14 19:01:16 +01:00
6bde5492b6 [tv5unis] Add new extractor(closes #22399)(closes #24890) 2020-12-14 18:47:00 +01:00
6086df4d6a [videomore] add support more.tv (closes #27088) 2020-12-14 15:00:02 +01:00
c98052c5da [yandexmusic:album] Simplify 2020-12-14 12:52:25 +07:00
ab62bc5838 [yandexmusic] Add support for music.yandex.com (closes #27425) 2020-12-14 12:48:12 +07:00
bc87ba8424 [yandexmusic] DRY _VALID_URL base 2020-12-14 12:45:56 +07:00
b79df1b68d [nhk:program] Add support for audio programs and program clips 2020-12-13 22:04:44 +01:00
2797c7be45 [nhk] Add support for NHK video programs (#27230) 2020-12-13 20:57:19 +00:00
755f186e21 release 2020.12.14 2020-12-14 00:57:11 +07:00
2240a1dc4d [ChangeLog] Actualize
[ci skip]
2020-12-14 00:55:04 +07:00
03d3af9768 [test_InfoExtractor] PEP 8 2020-12-13 23:47:13 +07:00
5ce9527e16 [mdr] Bypass geo restriction 2020-12-13 23:43:16 +07:00
c527f5ada0 [mdr] Improve extraction (closes #24346, closes #26873) 2020-12-13 23:42:43 +07:00
ace52668f0 [PULL_REQUEST_TEMPLATE.md] Add tests requirement and make coding conventions requirement more explicit 2020-12-13 22:46:17 +07:00
9c33eb027e [yandexmusic:album] Improve album title extraction (closes #27418) 2020-12-13 22:18:40 +07:00
679b711395 [eporner] Fix view count extraction and make optional (closes #23306) 2020-12-13 20:27:08 +07:00
1727541315 [extractor/common] Improve JSON-LD interaction statistic extraction (refs #23306) 2020-12-13 20:24:13 +07:00
45b0a0d11b [eporner] Fix embed test URL 2020-12-13 19:57:11 +07:00
e665fcd4da [eporner] Fix hash extraction and extend _VALID_URL (#27396)
Co-authored-by: Sergey M <dstftw@gmail.com>
2020-12-13 19:56:28 +07:00
aae737d4af [slideslive] use m3u8 entry protocol for m3u8 formats(closes #27400) 2020-12-13 12:43:21 +01:00
92a6de861e [downloader/hls] delegate manifests with media initialization to ffmpeg 2020-12-13 12:40:10 +01:00
5ff881aee6 [twitcasting] fix format extraction and improve info extraction(closes #24868) 2020-12-13 12:13:58 +01:00
eae19a4473 [extractor/common] Document duration meta field for playlists 2020-12-13 16:53:23 +07:00
f70c263ce5 [linuxacademy] Fix authentication and extraction (closes #21129, closes #26223, closes #27402) 2020-12-13 16:52:36 +07:00
92d135921f [itv] clean description from HTML tags (closes #27399) 2020-12-13 09:51:03 +01:00
d8008dee4f [vlive] Sort live formats (closes #27404) 2020-12-13 15:04:18 +07:00
bb38a12157 [hotstart] fix and improve extraction
- fix format extraction (closes #26690)
- extract thumbnail URL (closes #16079, closes #20412)
- support country specific playlist URLs (closes #23496)
- select the last id in video URL (closes #26412)
2020-12-12 23:07:52 +01:00
bcc8ef0a5a [youtube] Add some invidious instances (#27373)
Co-authored-by: Cássio <heyitscassio@cock.li>
2020-12-13 01:27:56 +07:00
3cb1a5dc73 release 2020.12.12 2020-12-12 07:09:59 +07:00
ed40c02c9b [ChangeLog] Actualize
[ci skip]
2020-12-12 07:04:28 +07:00
b5fdceb4d1 [ruutu] Extract more metadata and detect non-free videos (closes #21154) 2020-12-12 06:49:43 +07:00
1e6f7f3b45 [ruutu] Authenticate format URLs (closes #21031, closes #26782) 2020-12-12 05:57:14 +07:00
469ff02f4e [ruutu] Add support for static.nelonenmedia.fi (closes #25412) 2020-12-12 05:10:54 +07:00
56f1c5ad38 [ruutu] Extend _VALID_URL (closes #24839) 2020-12-12 05:10:54 +07:00
3a6b8f4edf [facebook] Add support archived live video URLs(closes #15859) 2020-12-11 23:09:42 +01:00
3ae02d4a62 [wdr] Extent subtitles extraction and improve overall extraction (closes #22672, closes #22723) 2020-12-12 04:52:35 +07:00
59fea63dc2 [facebook] add support for videos attached to Relay based story pages(#10795) 2020-12-11 22:44:18 +01:00
22933e02d4 [wdr:page] Add support for kinder.wdr.de (closes #27350) 2020-12-12 04:27:25 +07:00
98a62d7cbd [facebook] Add another regex for handleServerJS 2020-12-11 21:50:40 +01:00
d6c5fdef34 [facebook] fix embed page extraction 2020-12-11 21:27:47 +01:00
4b9051cf39 [YoutubeDL] Improve thumbnails' filenames deducing (closes #26010) (#27244) 2020-12-12 03:12:31 +07:00
00f5068908 [facebook] add support for Relay post pages(closes #26935) 2020-12-11 17:49:22 +01:00
432c6b0f48 [facebook] proper support for watch videos(closes #22795)(#27062) 2020-12-11 16:05:47 +01:00
ad5e5788ff Revert "[facebook] add support for watch videos(closes #22795)"
This reverts commit dc65041c22.
2020-12-11 15:54:24 +01:00
dc65041c22 [facebook] add support for watch videos(closes #22795) 2020-12-11 13:54:22 +01:00
9f88b07945 [facebook] add support for group posts with multiple videos(closes #19131) 2020-12-11 12:04:02 +01:00
225646c4ca [itv] remove old extractio method and fix series metadata extraction
closes #23177
closes #26897
2020-12-10 22:28:44 +01:00
5c6e9f0b6c [facebook] redirect Mobile URLs to Desktop URLs
closes #24831
closes #25624
2020-12-10 15:19:43 +01:00
6c370bc149 [facebook] Add support for Relay based pages(closes #26823) 2020-12-10 14:52:35 +01:00
37fd242342 [facebook] try to reduce unessessary tahoe requests 2020-12-10 12:10:33 +01:00
aee1f87168 [facebook] remove hardcoded chrome user-agent
closes #18974
closes #25411
closes #26958
closes #27329
2020-12-09 23:54:15 +01:00
b69bb1ed11 [smotri] Remove extractor (#27358) 2020-12-10 02:01:23 +07:00
772cefef8c [ChangeLog] Actualize
[ci skip]
2020-12-10 00:12:59 +07:00
842654b6d0 [README.md] Update travis CI tld (#27354) 2020-12-09 21:50:42 +07:00
df5e50954b [beampro] Remove Extractor
closes #17290
closes #22871
closes #23020
closes #23061
closes #26099
2020-12-09 10:41:08 +01:00
a4a2fa8754 release 2020.12.09 2020-12-09 04:25:24 +07:00
9da0504a09 [ChangeLog] Actualize
[ci skip]
2020-12-09 04:24:31 +07:00
470cf496f5 [youtube:tab] Improve identity token extraction (closes #27197) 2020-12-09 04:17:47 +07:00
e029da9add [youtube:tab] Make click tracking params on continuation optional 2020-12-09 04:06:44 +07:00
e00b8f60d4 [youtube:tab] Delegate inline playlists to tab-based playlists (closes #27298) 2020-12-09 03:40:02 +07:00
644c3ef886 [tubitv] Extract release year (#27317) 2020-12-09 01:14:46 +07:00
9d8d0f8b4a [youtube] Remove unused code (#27343) 2020-12-09 00:41:47 +07:00
5a1fbbf8b7 [extractor/common] Fix inline HTML5 media tags processing and add test (closes #27345) 2020-12-09 00:05:21 +07:00
e2bdf8bf4f [amcnetworks] Fix free content extraction(closes #20354) 2020-12-08 17:11:28 +01:00
c368dc98e0 [lbry] add support for channel extraction(closes #25584) 2020-12-08 14:53:22 +01:00
e7eff914cd [lbry] add support for short and embed URLs and fix channel metadata extraction 2020-12-08 10:52:52 +01:00
07333d0062 [telequebec] Fix Extraction and Add Support for video.telequebec.tv
closes #25733
closes #26883
closes #27339
2020-12-08 00:35:47 +01:00
5bd7ad2e81 [youtube:tab] Capture and output alerts (closes #27340) 2020-12-08 01:12:00 +07:00
3ded751985 [generic] comment a test covered now by AmericasTestKitchenIE 2020-12-07 15:27:13 +01:00
6956db3606 [tvplay:home] Fix extraction(closes #21153) 2020-12-07 15:12:54 +01:00
17b01228f8 [americastestkitchen] Fix Extraction and add support for Cook's Country and Cook's Illustrated
closes #17234
closes #27322
2020-12-06 23:41:18 +01:00
4f1ecca58d [slideslive] Add support for yoda service videos and extract subtitles (closes #27323) 2020-12-07 03:27:21 +07:00
2717036489 release 2020.12.07 2020-12-07 02:03:34 +07:00
d9482c0083 [ChangeLog] Actualize
[ci skip]
2020-12-07 02:01:53 +07:00
791b743765 [extractor/generic] Remove unused import 2020-12-07 02:01:47 +07:00
fa604d9083 [aenetworks] Fix extraction
- Fix Fastly format extraction
- Add support for play and watch subdomains
- Extract series metadata

closes #23363
closes #23390
closes #26795
closes #26985
2020-12-06 19:50:05 +01:00
2bf0634d16 [youtube] Improve youtu.be extraction in non-existing playlists (closes #27324) 2020-12-07 01:48:10 +07:00
dccf4932e1 [extractor/common] Extract timestamp from Last-Modified header 2020-12-07 01:15:30 +07:00
91dd25fe1e [extractor/common] Add support for dl8-* media tags (closes #27283) 2020-12-07 01:08:22 +07:00
06bf2ac20f [extractor/common] Eliminate media tag name regex duplication 2020-12-07 00:56:29 +07:00
6ad0d8781e [extractor/common] Fix media type extraction for HTML5 media tags in start/end form 2020-12-07 00:45:16 +07:00
f2c704e112 [generic] Extract RSS video itunes metadata 2020-12-06 23:08:03 +07:00
5e822c2526 [generic] Extract RSS video timestamp 2020-12-06 22:30:30 +07:00
cc017e07ca [generic] Extract RSS video description (#27177) 2020-12-06 22:12:25 +07:00
082da36416 [nrk] reduce requests for Radio series 2020-12-05 16:24:49 +01:00
6bf95b15ee [nrk] reduce the number of instalments requests 2020-12-05 15:35:29 +01:00
4c93b2fd15 [nrk] improve format extraction 2020-12-05 09:13:42 +01:00
1b26bfd425 [nrk] improve extraction
- improve format extraction for old akamai formats
- update some of the tests
- add is_live value to entry info dict
- request instalments only when their available
- fix skole extraction
2020-12-05 00:40:03 +01:00
13ec444a98 [peertube] Extract fps 2020-12-05 04:45:30 +07:00
51579d87e4 [peertube] Recognize audio-only formats (closes #27295) 2020-12-05 04:41:33 +07:00
e147619669 release 2020.12.05 2020-12-05 03:40:00 +07:00
e7f93fbd85 [ChangeLog] Actualize
[ci skip]
2020-12-05 03:31:07 +07:00
58f7ada235 [teachable:course] Improve extraction (closes #24507, closes #27286) 2020-12-05 03:24:56 +07:00
c67b33888f [nrk] Improve error extraction 2020-12-05 03:00:10 +07:00
e8c0af04b7 [nrktv] Relax _VALID_URL 2020-12-05 02:46:28 +07:00
5d769860c3 [nrktv:series] Improve extraction (closes #21926) 2020-12-05 02:43:33 +07:00
02b04785ee [nrktv:series] Improve extraction 2020-12-05 02:21:07 +07:00
41c92b8d02 [nrktv:season] Improve extraction 2020-12-05 01:20:14 +07:00
fe0c28f956 [nrk] fix call to moved method 2020-12-04 18:08:08 +01:00
957c65b9ea [nrk] fix typo 2020-12-04 18:05:27 +01:00
5e95e18ce9 [nrk] improve format extraction and geo-restriction detection (closes #24221) 2020-12-04 18:04:38 +01:00
e91df0c550 [pornhub] Handle HTTP errors gracefully (closes #26414) 2020-12-04 23:56:50 +07:00
c5636e9bca [nrktv] Relax _VALID_URL (closes #27299, closes #26185) 2020-12-04 23:38:42 +07:00
2e47264235 [zdf] extract webm formats(closes #26659) 2020-12-04 14:16:32 +01:00
1c78cb118c [travis] Disable download jobs
Until youtube-dl OSS approval by Travis or moving to GitHub Actions
[ci skip]
2020-12-03 23:25:36 +07:00
beab2f88c9 [nhk] Add audio clip test to NHK extractor (#27269) 2020-12-03 07:47:56 +00:00
0025447369 [gamespot] Extract DASH and HTTP formats 2020-12-03 00:35:47 +01:00
da4304609d [extractor/commons] improve Akamai HTTP formats extraction 2020-12-03 00:33:55 +01:00
ea89680aea [tver] correct episode_number key 2020-12-02 22:49:51 +01:00
664dd8ba85 [extractor/common] improve Akamai HTTP format extraction
- Allow m3u8 manifest without an additional audio format
- Fix extraction for qualities starting with a number
Solution provided by @nixxo based on: https://stackoverflow.com/a/5984688
2020-12-02 21:49:09 +01:00
64554c12e1 [tver] Add new extractor (closes #26662)(closes #27284) 2020-12-02 21:49:09 +01:00
4ded9c0f00 [pornhub] Add support for pornhub.org (#27276)
Most ISPs block the other two TLDs through deep packet inspection
2020-12-03 01:30:08 +07:00
c0820dd52a [extractors] Add QubIE import 2020-12-02 00:41:21 +01:00
2bb70750a9 release 2020.12.02 2020-12-02 01:37:40 +07:00
09d923f2eb [ChangeLog] Actualize
[ci skip]
2020-12-02 01:22:43 +07:00
37d979ad33 [tva] Add support for qub.ca (closes #27235) 2020-12-01 12:25:02 +01:00
95ac4de229 [toggle] Detect DRM protected videos (closes #16479)(closes #20805) 2020-12-01 10:38:53 +01:00
d3e142b3fa [toggle] Add support for new MeWatch URLs (closes #27256) 2020-12-01 10:13:06 +01:00
132aece1ed [youtube:tab] Extract channels only from channels tab (closes #27266) 2020-12-01 04:44:10 +07:00
3e4e338133 [cspan] Extract info from jwplayer data (closes #3672, closes #3734, closes #10638, closes #13030, closes #18806, closes #23148, closes #24461, closes #26171, closes #26800, closes #27263) 2020-12-01 01:53:12 +07:00
be19ae11fd [cspan] Pass Referer header with format's video URL (#26032) (closes #25729) 2020-12-01 01:14:29 +07:00
59d63d8d4a [youtube] Improve age-gated videos extraction (closes #27259) 2020-12-01 00:49:26 +07:00
cfeba5d17f [mediaset] add support for movie URLs(closes #27240) 2020-11-30 09:46:15 +01:00
6da0e5e7a2 [yandexmusic] Refactor and add support for artist's tracks and albums (closes #11887, closes #22284) 2020-11-30 00:25:06 +07:00
d6ce649f15 [yandexmusic:track] Fix extraction (closes #26449, closes #26669, closes #26747, closes #26748, closes #26762) 2020-11-29 22:15:51 +07:00
b449b73dcc release 2020.11.29 2020-11-29 13:53:01 +07:00
16c822e91e [ChangeLog] Actualize
[ci skip]
2020-11-29 13:49:12 +07:00
4318170779 [drtv] Extend _VALID_URL (#27243) 2020-11-29 13:44:36 +07:00
fb626c0586 [tiktok] Fix extraction (closes #20809, closes #22838, closes #22850, closes #25987, closes #26281, closes #26411, closes #26639, closes #26776, closes #27237) 2020-11-29 08:09:20 +07:00
717d1d2d5a [ina] Add support for mobile URLs (#27229) 2020-11-29 04:15:53 +07:00
9585b376db [YoutubeDL] Write static debug to stderr and respect quiet for dynamic debug (closes #14579, closes #22593)
TODO: logging and verbosity needs major refactoring (refs #10894)
2020-11-29 04:04:06 +07:00
f04cfe24e0 [pornhub] Fix like and dislike count extraction (closes #27227) (#27234) 2020-11-29 02:32:13 +07:00
20c50c6556 [youtube] Improve yt initial player response extraction (closes #27216) 2020-11-28 15:02:31 +07:00
f9f9699f2f [videa] improve extraction 2020-11-26 12:56:49 +01:00
a3cf22e590 [videa] Adapt to updates (#26301)
closes #25973, closes #25650.
2020-11-26 11:55:06 +00:00
99de2f38d3 [spreaker] fix SpreakerShowIE test URL 2020-11-25 21:39:17 +01:00
9fe50837c3 release 2020.11.26 2020-11-26 03:05:51 +07:00
4dc545553f [ChangeLog] Actualize
[ci skip]
2020-11-26 03:03:51 +07:00
686e898fde [spreaker] Add extractor (closes #13480, closes #13877) 2020-11-26 02:58:48 +07:00
3a78198a96 [vlive] improve extraction for geo-restricted videos 2020-11-25 11:40:37 +01:00
836c810716 [vlive] Add support for post URLs(closes #27122)(closes #27123) 2020-11-25 11:26:26 +01:00
97c5be383c [viki] fix video API request(closes #27184) 2020-11-24 23:25:03 +01:00
a7ea88537a [bbc] fix BBC Three clip extraction 2020-11-24 22:54:08 +01:00
6c35de4c6b [bbc] fix BBC News videos extraction 2020-11-24 22:49:04 +01:00
579d43951d [medaltv] improve extraction 2020-11-24 18:29:46 +01:00
e24ebeafd1 [medaltv] Add new extractor (#27149) 2020-11-24 18:27:33 +01:00
01c92973dd [youtube] imporve music metadata and license extraction(closes #26013) 2020-11-23 22:44:59 +01:00
f4415faa46 [downloader/fragment] Set final file's mtime according to last fragment's Last-Modified header (closes #11718, closes #18384, closes #27138) 2020-11-24 04:17:24 +07:00
a86ce9d7a1 [nrk] Fix extraction 2020-11-24 02:59:49 +07:00
37258c644f [cda] Fix extraction (closes #17803, closes #24458, closes #24518, closes #26381) 2020-11-24 02:18:40 +07:00
d0512ac4c5 release 2020.11.24 2020-11-24 00:23:14 +07:00
da4eaa15a2 [ChangeLog] Actualize
[ci skip]
2020-11-24 00:20:42 +07:00
191286265d [youtube:tab] Fix feeds extraction (closes #25695, closes #26452) 2020-11-24 00:10:25 +07:00
3234272818 [youtube:tab] PEP 8 2020-11-23 22:34:27 +07:00
9d2c90354f [youtube:favorites] Restore extractor 2020-11-23 22:33:21 +07:00
316b10855a [youtube:tab] Fix some weird typo (closes #27157) 2020-11-23 22:19:25 +07:00
484fe78737 [pinterest] Add support for large collections(more than 25 pins) 2020-11-23 14:16:38 +01:00
2cd43a00d1 [franceinter] flake8 2020-11-22 19:38:45 +01:00
dd0f524c69 [franceinter] add thumbnail url (#27153)
Co-authored-by: remitamine <remitamine@gmail.com>
2020-11-22 19:35:53 +01:00
c84f9475b8 [box] Add new extractor(#5949) 2020-11-22 17:39:41 +01:00
15f2734791 [nytimes] Add new cooking.nytimes.com extractor (#27143)
* [nytimes] support cooking.nytimes.com, resolves #27112

Co-authored-by: remitamine <remitamine@gmail.com>
2020-11-22 14:12:47 +01:00
cb6e24f946 [lbry] relax _VALID_URL regex(closes #27144) 2020-11-22 13:16:03 +01:00
9d531aa291 [rumble] add support for embed pages(#10785) 2020-11-22 13:07:04 +01:00
e9cbb98a0f [skyit] add support for multiple Sky Italia websites(closes #26629) 2020-11-22 13:01:56 +01:00
193422e12a [extractor/common] add generic support for akamai http format extraction 2020-11-22 12:54:55 +01:00
c4cabf040e [pinterest] Add extractor (closes #25747) 2020-11-22 05:04:01 +07:00
f23eceebbf release 2020.11.21.1 2020-11-21 23:59:11 +07:00
650bd8f623 [ChangeLog] Actualize
[ci skip]
2020-11-21 23:56:32 +07:00
5d8cb4367d release 2020.11.21 2020-11-21 23:54:33 +07:00
82abc13aed [youtube:tab] Comment out test 2020-11-21 23:41:49 +07:00
1e72660c9b [svtplay] Fix test title 2020-11-21 23:36:25 +07:00
049f224248 [svtplay] Add support for svt.se/barnkanalen (closes #24817) 2020-11-21 23:35:32 +07:00
0ada1b90b8 [svt] Extract timestamp and thumbnail in more cases (#27130)
Add timestamp, set to "valid from" which i think could been seen as publish time.
Add thumbnail in more cases, seems to was only done in the embedded data case for some reason.
Switch svtplay test url to an existing video and also one with no expire date.
Also add an additional thumbnail url test regex.
2020-11-21 23:24:37 +07:00
71ddc222ad Fix typos (#27084)
* spelling: authorization

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: brightcove

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: creation

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: exceeded

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: exception

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: extension

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: extracting

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: extraction

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: frontline

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: improve

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: length

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: listsubtitles

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: multimedia

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: obfuscated

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: partitioning

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: playlist

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: playlists

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: restriction

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: services

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: split

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: srmediathek

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: support

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: thumbnail

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: verification

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: whitespaces

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-11-21 22:00:05 +07:00
21292c0649 [youtube] Fix error reason extraction (#27081) 2020-11-21 21:52:20 +07:00
46a265a2da [youtube] Fix like and dislike count extraction (closes #25977) 2020-11-21 20:49:56 +07:00
e2096776b9 [youtube:tab] Add support for current video and fix lives extraction (closes #27126) 2020-11-21 20:48:13 +07:00
a1c88c4819 [infoq] fix format extraction(closes #25984) 2020-11-20 23:23:55 +01:00
51ca93d751 [francetv] Update to fix thumbnail URL issue (#27120)
Fix the thumbnail URL. The issue was here for many years, never fixed. It's done ! :-)

Example : https://www.france.tv/france-2/de-gaulle-l-eclat-et-le-secret/de-gaulle-l-eclat-et-le-secret-saison-1/2035247-solitude.html

failed thumbnail url generated : http://pluzz.francetv.fr/staticftv/ref_emissions/2020-11-02/EMI_1104da66f533cc7dc5d0d07a181a18c2e2fe1d81_20201014122553940.jpg

right thumbnail url fixed : https://sivideo.webservices.francetelevisions.fr/staticftv/ref_emissions/2020-11-02/EMI_1104da66f533cc7dc5d0d07a181a18c2e2fe1d81_20201014122553940.jpg
2020-11-20 22:44:08 +01:00
a7e0531999 [downloader/http] Fix crash during urlopen caused by missing reason of URLError 2020-11-21 04:22:21 +07:00
ab0eda99e1 [YoutubeDL] Fix --ignore-errors for playlists with generator-based entries of url_transparent (closes #27064) 2020-11-21 04:00:08 +07:00
ec99f47108 [youtube:tab] Replace some test URLs with RIAA-friendly ones 2020-11-20 23:34:46 +07:00
b31b5f4434 [youtube] Improve yt initial data extraction (closes #27093) 2020-11-20 23:21:52 +07:00
86f2fa1590 [discoverynetworks] add support new TLC/DMAX URLs(closes #27100) 2020-11-20 10:47:52 +01:00
7bc7fbce23 [rai] fix protocol relative relinker URLs(closes #22766) 2020-11-20 10:26:55 +01:00
a78e530c14 [rai] fix unavailable video format detection 2020-11-20 10:10:57 +01:00
af7bb684c0 [rai] improve extraction 2020-11-20 10:01:56 +01:00
dd9e0f58f3 [rai] Fix extraction for recent raiplay.it updates (#27077)
- Remove first test of RaiPlayIE: it is no longer available
- Make RaiPlayIE extension-agnostic (passing possible `.json' URLs is now
  supported too)
- Adjust RaiPlayLiveIE to recent raiplay.it updates.  Passing it as
  `url_transparent' is no longer supported (there is no longer an accessible
  ContentItem)
- Adjust RaiPlayPlaylistIE to recent raiplay.it updates and instruct it about
  ContentSet-s.
- Update a RaiIE test and remove two tests that are no longer availables

Thanks to @remitamine for the review!
2020-11-20 10:00:05 +01:00
59e583f7e8 [viki] improve format extraction 2020-11-19 22:49:28 +01:00
daa25d4142 [viki] fix stream extraction from mpd (#27092)
Co-authored-by: beefchop <beefchop@users.noreply.github.com>
2020-11-19 21:38:09 +01:00
25a35cb38a [googledrive] fix format extraction(closes #26979) 2020-11-19 20:01:24 +01:00
2cf8003638 [amara] improve extraction 2020-11-19 17:29:30 +01:00
cf1a8668e8 [Amara] Add new extractor (#20618)
* [Amara] Add new extractor
2020-11-19 17:26:53 +01:00
2dbb45ae82 [vimeo:album] fix extraction(closes #27079) 2020-11-19 13:13:33 +01:00
4fe190df70 [mtv] fix mgid extraction(closes #26841) 2020-11-19 11:57:09 +01:00
039e715b30 release 2020.11.19 2020-11-19 05:22:27 +07:00
32152bab7a [ChangeLog] Actualize
[ci skip]
2020-11-19 05:21:09 +07:00
b1347a5881 [youporn] Fix upload date extraction and make comment count optional (closes #26986) 2020-11-19 05:16:25 +07:00
91e954587f [arte] Rework extractors
* Reimplement embed and playlist extractors to delegate to the single entrypoint artetv extractor
  Beware reluctant download archive extractor keys breakage.
* Improve embeds detection (closes #27057)
- Remove obsolete code
2020-11-19 05:02:04 +07:00
9b505185da [arte] Extract m3u8 formats (closes #27061) 2020-11-19 03:26:49 +07:00
444a68e0ec [mgtv] fix format extraction(closes #26415) 2020-11-18 21:07:00 +01:00
c7178f0f7a [extractor/common] Output error for invalid URLs in _is_valid_url (refs #21400, refs #24151, refs #25617, refs #25618, refs #25586, refs #26068, refs #27072) 2020-11-18 23:31:35 +07:00
5c3f7014ef [lbry] add support for odysee.com domain(closes #26806) 2020-11-18 00:41:07 +01:00
d65628ef03 [francetv] improve info extraction 2020-11-17 23:16:04 +01:00
8a6c5b0806 [francetv] Add fallback video url extraction (#27047)
Fallback on another API endpoint when no video formats are found.

Closes ytdl-org#22561
2020-11-17 23:06:19 +01:00
9360936f26 release 2020.11.18 2020-11-18 04:15:30 +07:00
bb2b89e077 [ChangeLog] Actualize
[ci skip]
2020-11-18 04:11:58 +07:00
2d7a29081c [spiegel] Fix extraction (closes #24206, closes #24767)
Code picked from PR #24767 since original repo is not available due to takedown.
2020-11-18 04:09:02 +07:00
2864179293 [youtube] Improve extraction
+ Add support for --no-playlist (closes #27009)
* Improve playlist and mix extraction (closes #26390, closes #26509, closes #26534, closes #27011)
+ Extract playlist uploader data
* Update tests
2020-11-18 03:34:08 +07:00
284f8306df [youtube:tab] fix view_count extraction(closes #27051) 2020-11-17 20:32:50 +01:00
aa613ef7e1 [malltv] fix extraction(closes #27035) 2020-11-17 19:13:38 +01:00
cb2b9a22a5 [bandcamp] extract playlist_description(closes #22684) 2020-11-17 14:46:02 +01:00
5b867c15a8 [urplay] fix extraction(closes #26828) 2020-11-17 13:12:20 +01:00
f8c749f12c [youtube:tab] Fix playlist title extraction (closes #27015) 2020-11-17 07:01:41 +07:00
7d509c613b [youtube] Fix chapters extraction (closes #26005) 2020-11-17 04:28:35 +07:00
b92e95aa01 release 2020.11.17 2020-11-17 03:59:54 +07:00
e3cad6bd99 [ChangeLog] Actualize
[ci skip]
2020-11-17 03:50:12 +07:00
6699b6ce41 [youtube:tab] Fix extraction with cookies provided (closes #27005) 2020-11-17 03:40:57 +07:00
91dcde8a38 [lrt] fix extraction with empty tags(closes #20264) 2020-11-16 21:28:12 +01:00
11f3471c4b [ndr:embed:base] Extract subtitles (closes #25447, closes #26106) 2020-11-17 03:24:23 +07:00
f4093b34f6 [servus] Add support for pm-wissen.com (closes #25869) 2020-11-17 03:02:07 +07:00
a80b23c373 [servus] Fix extraction (closes #26872, closes #26967, closes #26983, closes #27000) 2020-11-17 02:57:26 +07:00
2e7fa18bb9 [xtube] Fix extraction (closes #26996) 2020-11-17 01:32:48 +07:00
fe07e788bf [utils] Skip ! prefixed code in js_to_json 2020-11-17 01:30:43 +07:00
6d3bdcf217 [lrt] fix extraction 2020-11-16 19:17:10 +01:00
ec2a2ab441 [lbry] Add new extractor 2020-11-16 19:04:55 +01:00
9448a20312 [condenast] fix extraction and extract subtitles 2020-11-16 18:57:56 +01:00
3f1748b944 [bandcamp] fix extraction 2020-11-16 17:26:09 +01:00
fe13087cd1 [rai] fix RaiPlay extraction 2020-11-16 17:17:52 +01:00
f22fa82d7f [extractors] Remove VLivePlaylistIE import 2020-11-16 16:36:56 +01:00
059fa9aa81 [vlive] fix extraction 2020-11-16 16:35:08 +01:00
650aec4a98 [usanetwork] fix extraction 2020-11-16 16:21:48 +01:00
2ea9c97432 [nbc] fix NBCNews/Today/MSNBC extraction 2020-11-16 16:18:37 +01:00
efc589b865 [devscripts/make_lazy_extractors] Correct a spelling mistake (#26991) 2020-11-16 22:08:20 +07:00
1737ea69b9 [cnbc] fix extraction 2020-11-16 15:55:07 +01:00
1fb034d029 [youtube] Remove RIAA copyrighted media from tests as per [1]
1. Github dmca and 1201 notice re youtube-dl 9-21-20
2020-11-16 21:08:16 +07:00
28f9568a84 release 2020.11.12 2020-11-12 06:23:46 +07:00
5bbdadd5f8 [ChangeLog] Actualize
[ci skip]
2020-11-12 06:18:16 +07:00
2de2ca6659 [youtube] Rework extractors
WIP
2020-11-12 06:16:37 +07:00
051071203c release 2020.11.01.1 2020-11-01 08:58:40 +07:00
34299510bb release 2020.11.01 2020-11-01 08:52:27 +07:00
b9bceba37c [ChangeLog] Actualize
[ci skip]
2020-11-01 07:34:20 +07:00
6d4733ce7b [youtube] Fix JS player URL extraction 2020-11-01 06:52:00 +07:00
416da574ec [ytsearch] Fix extraction (closes #26920) 2020-10-23 21:31:37 +07:00
48c5663c5f [afreecatv] Fix typo (#26970) 2020-10-22 19:15:05 +07:00
7d740e7dc7 [23video] Relax _VALID_URL (#26870) 2020-10-20 00:56:23 +07:00
4eda10499e [utils] Don't attempt to coerce JS strings to numbers in js_to_json (#26851)
The current logic in `js_to_json` tries to rewrite octal/hex numbers to
decimal. However, when the logic actually happens the `"` or `'` have
already been trimmed off. This causes what were originally strings, that
happen to look like octal/hex numbers, to get rewritten to decimal and
returned as a number rather than a string.

In practive something like:

```js
{
  "0x40": "foo",
  "040": "bar",
}
```

would get rewritten as:

```json
{
  64: "foo",
  32: "bar
}
```

This is problematic since this isn't valid JSON as you cannot have
non-string keys.
2020-10-18 00:10:41 +07:00
605535776a [ustream] Add support for video.ibm.com (#26894) 2020-10-17 23:14:46 +07:00
1050e0d09f [iqiyi] Fix typo (#26884) 2020-10-17 23:02:17 +07:00
d65d89183f [expressen] Add support for di.se (closes #26670) 2020-09-24 07:37:10 +07:00
0c92f1e96b [iprima] Improve video id extraction (#26507) (closes #26494) 2020-09-24 06:46:58 +07:00
adae9e844b [README.md] Fix autonumber sequence description (refs #26686) 2020-09-24 06:36:07 +07:00
c5764b3f89 [downloader/http] Properly handle missing message in SSLError (closes #26646) 2020-09-22 07:01:59 +07:00
0837992a22 [downloader/http] Fix access to not yet opened stream in retry 2020-09-22 06:44:14 +07:00
b55715934b release 2020.09.20 2020-09-20 12:30:45 +07:00
bbc3b5b4bb [ChangeLog] Actualize
[ci skip]
2020-09-20 12:24:32 +07:00
1ca5f821c8 [redtube] Extend _VALID_URL (#26506) 2020-09-20 11:39:42 +07:00
defc820b70 [twitch] Switch streams to GraphQL and refactor (closes #26535) 2020-09-20 10:05:00 +07:00
82ef02e936 [telequebec] Fix issues (closes #26368) 2020-09-19 07:56:00 +07:00
b856b3997c [telequebec] Add support for brightcove videos (closes #25833) 2020-09-19 07:52:57 +07:00
cd85a1bb8b [pornhub] Extract metadata from JSON-LD (closes #26614) 2020-09-19 06:34:34 +07:00
ce5b904050 [extractor/common] Relax interaction count extraction in _json_ld 2020-09-19 06:33:17 +07:00
ad06b99dd4 [extractor/common] Extract author as uploader for VideoObject in _json_ld 2020-09-19 06:13:42 +07:00
540b9f5164 [pornhub] Fix view count extraction (#26621) (refs #26614) 2020-09-19 05:59:19 +07:00
6e65a2a67e [downloader/hls] Fix incorrect end byte in Range HTTP header for media segments with EXT-X-BYTERANGE (#24512) (closes #14748)
The end of the byte range is the first byte that is NOT part of the to
be downloaded range. So don't include it into the requested HTTP
download range, as this additional byte leads to a broken TS packet and
subsequently to e.g. visible video corruption.

Fixes #14748.
2020-09-18 05:26:56 +07:00
f8c7bed133 [extractor/common] Handle ssl.CertificateError in _request_webpage (closes #26601)
ssl.CertificateError is raised on some python versions <= 3.7.x
2020-09-18 03:41:16 +07:00
cdc55e666f [downloader/http] Improve timeout detection when reading block of data (refs #10935) 2020-09-18 03:32:54 +07:00
86b7c00adc [downloader/http] Retry download when urlopen times out (#26603) (refs #10935) 2020-09-18 03:15:44 +07:00
e8c5d40bc8 release 2020.09.14 2020-09-14 03:37:36 +07:00
ca7ebc4e5e [ChangeLog] Actualize
[ci skip]
2020-09-14 03:35:18 +07:00
bff857a8af [postprocessor/embedthumbnail] Fix issues (closes #25717)
* Fix WebP with wrong extension processing
* Fix embedding of thumbnails with % character in path
2020-09-14 03:28:31 +07:00
a31a022efd [postprocessor/embedthumbnail] Add support for non jpeg/png thumbnails (closes #25687) 2020-09-14 03:10:01 +07:00
45f6362464 [rtlnl] Extend _VALID_URL for new embed URL schema 2020-09-13 21:42:06 +07:00
97f34a48d7 [rtlnl] Extend _VALID_URL (#26549) (closes #25821) 2020-09-13 21:38:16 +07:00
ea74e00b3a [youtube] Fix empty description extraction (#26575) (closes #26006) 2020-09-13 21:23:21 +07:00
06cd4cdb25 [srgssr] Extend _VALID_URL (closes #26555, closes #26556, closes #26578) 2020-09-13 21:07:25 +07:00
da2069fb22 [googledrive] Use redirect URLs for source format (closes #18877, closes #23919, closes #24689, closes #26565) 2020-09-13 20:49:32 +07:00
95c9810015 [svtplay] Fix id extraction (closes #26576) 2020-09-13 18:59:37 +07:00
b03eebdb6a [redbulltv] improve support for rebull.com TV localized URLS(#22063) 2020-09-13 11:26:11 +01:00
1f7675451c [redbulltv] Add support for new redbull.com TV URLs(closes #22037)(closes #22063) 2020-09-12 19:27:58 +01:00
aa27253556 [soundcloud] Reduce pagination limit to fix 502 Bad Gateway errors when listing a user's tracks. (#26557)
Per the documentation here https://developers.soundcloud.com/blog/offset-pagination-deprecated the maximum limit is 200, so let's respect that (even if a higher value sometimes works).

Co-authored-by: tfvlrue <tfvlrue>
2020-09-12 09:35:11 +00:00
d51e23d9fc release 2020.09.06 2020-09-06 13:00:41 +07:00
6cd452acff [ChangeLog] Actualize
[ci skip]
2020-09-06 12:57:56 +07:00
50e9fcc1fd [nrktv:episode] Improve video id extraction (closes #25594, closes #26369, closes #26409) 2020-09-06 12:43:50 +07:00
16ee69c1b7 [youtube] Fix age gate content detection (#26100) (closes #26152, closes #26311, closes #26384) 2020-09-06 11:44:53 +07:00
67171ed7e9 [youtube:user] Extend _VALID_URL (closes #26443) 2020-09-06 11:31:28 +07:00
1d9bf655e6 [utils] Recognize wav mimetype (closes #26463) 2020-09-06 11:19:53 +07:00
62ae19ff76 [xhamster] Improve initials regex (#26526) (closes #26353) 2020-09-06 11:10:27 +07:00
5ed05f26ad [svtplay] Fix svt id extraction (closes #26425, closes #26428, closes #26438) 2020-09-06 10:45:57 +07:00
841b683804 [twitch] Rework extractors (closes #12297, closes #20414, closes #20604, closes #21811, closes #21812, closes #22979, closes #24263, closes #25010, closes #25553, closes #25606)
* Switch to GraphQL.
+ Add support for collections.
+ Add support for clips and collections playlists.
2020-09-06 10:45:34 +07:00
f5863a3ea0 [biqle] improve video_ext extraction 2020-08-27 19:20:41 +01:00
10709fc7c6 [xhamster] Extend _VALID_URL (closes #25927) 2020-08-12 21:51:50 +07:00
a7e348556a [xhamster] Fix extraction (closes #26157) (#26254) 2020-08-12 21:42:17 +07:00
6cb30ea5ed [xhamster] Extend _VALID_URL (closes #25789) (#25804) 2020-08-12 21:37:22 +07:00
a4ed50bb84 release 2020.07.28 2020-07-28 05:13:03 +07:00
570611955f [ChangeLog] Actualize
[ci skip]
2020-07-28 05:07:54 +07:00
e450f6cb63 [youtube] Fix sigfunc name extraction (closes #26134, closes #26135, closes #26136, closes #26137) 2020-07-28 05:05:38 +07:00
a115e07594 [youtube] Improve description extraction (closes #25937) (#25980) 2020-07-14 12:01:15 +01:00
718393c632 [wistia] Restrict embed regex (closes #25969) 2020-07-11 18:27:19 +07:00
07af16b92e [youtube] Prevent excess HTTP 301 (#25786) 2020-07-01 02:56:16 +07:00
e942cfd1a7 [youtube:playlists] Extend _VALID_URL (closes #25810) 2020-06-28 10:30:03 +07:00
9a7e5cb88a [bellmedia] add support for cp24.com clip URLs(closes #25764) 2020-06-23 15:09:13 +01:00
2391941f28 [brightcove] Improve embed detection (closes #25674) 2020-06-16 17:38:25 +07:00
9ff6165a81 release 2020.06.16.1 2020-06-16 06:22:01 +07:00
1c748722f9 [ChangeLog] Actualize
[ci skip]
2020-06-16 06:19:23 +07:00
ee0b726cd7 [youtube] Force old layout (closes #25682, closes #25683, closes #25680, closes #25686) 2020-06-16 06:17:53 +07:00
dbeafce5d5 [youtube] Fix categories and improve tags extraction 2020-06-16 03:13:39 +07:00
ed604ce7bc release 2020.06.16 2020-06-16 02:53:33 +07:00
7adc7ca547 [ChangeLog] Actualize
[ci skip]
2020-06-16 02:52:09 +07:00
a6211d237b [youtube] Fix uploader id and uploader URL extraction 2020-06-16 02:43:09 +07:00
7b16239a49 [youtube] Improve view count extraction 2020-06-16 02:38:45 +07:00
37357d21a9 [youtube] Fix upload date extraction 2020-06-16 02:37:19 +07:00
b477fc1314 [youtube] Fix thumbnails extraction and remove uploader id extraction warning (closes #25676) 2020-06-16 02:29:04 +07:00
d84b21b427 [youtube] Fix playlist and feed extraction (closes #25675) 2020-06-16 02:01:12 +07:00
48bd042ce7 [facebook] Support single-video ID links
I stumbled upon this at https://www.facebook.com/bwfbadminton/posts/10157127020046316 . No idea how prevalent it is yet.
2020-06-14 13:17:51 +02:00
84213ea8d4 [youtube] Extract chapters from JSON (closes #24819) 2020-06-06 04:22:10 +07:00
562de77f41 [kaltura] Add support for multiple embeds on a webpage (closes #25523) 2020-06-06 02:14:35 +07:00
e1723c4bac release 2020.06.06 2020-06-06 01:51:39 +07:00
607d204551 [ChangeLog] Actualize
[ci skip]
2020-06-06 01:49:27 +07:00
a5b6102ea8 [tele5] Bypass geo restriction 2020-06-06 01:45:05 +07:00
b77888228d [jwplatform] Add support for bypass geo restriction 2020-06-06 01:44:36 +07:00
0b1eaec3bc [tele5] Prefer jwplatform over nexx (closes #25533) 2020-06-06 01:35:09 +07:00
b37e47a3f9 [twitch:stream] Expect 400 and 410 HTTP errors from API 2020-06-06 00:57:40 +07:00
ce3735df02 [twitch:stream] Fix extraction (closes #25528) 2020-06-06 00:55:29 +07:00
a0455d0ffd [twitch] Pass v5 accept header and fix thumbnails extraction (closes #25531) 2020-06-06 00:12:47 +07:00
c8b232cc48 [brightcove] Sort imports 2020-06-05 23:35:57 +07:00
b4eb0bc7bd [brightcove] Fix subtitles extraction (closes #25540) 2020-06-05 23:33:14 +07:00
d5147b65ac [malltv] Add support for sk.mall.tv (#25445) 2020-06-01 21:11:31 +07:00
7b0b53ea69 [twitter:broadcast] Add untitled periscope broadcast test 2020-06-01 20:32:57 +07:00
7016e24ebe [periscope] Fix untitled broadcasts (#25482) 2020-06-01 20:31:51 +07:00
bef4688c72 [jwplatform] Improve embeds extraction (closes #25467) 2020-05-31 11:10:31 +07:00
228c1d685b release 2020.05.29 2020-05-29 03:33:13 +07:00
efd72b05d2 [ChangeLog] Actualize
[ci skip]
2020-05-29 03:28:44 +07:00
fe515e5c75 [ard:beta] Extend _VALID_URL (closes #25405) 2020-05-29 02:01:51 +07:00
1db5ab6b34 [youtube] Add support for more invidious instances (#25417) 2020-05-27 01:26:45 +07:00
2791e80b60 [postprocessor/ffmpeg] Embed series metadata with --add-metadata 2020-05-23 12:28:15 +07:00
8f841fafcd [giantbomb] Extend _VALID_URL (#25222) 2020-05-21 04:30:50 +07:00
a54c5f83c0 [ard] Improve _VALID_URL (closes #25134) (#25198) 2020-05-20 04:08:08 +07:00
cd13343ad8 [redtube] Improve formats extraction and extract m3u8 formats (closes #25311, closes #25321) 2020-05-20 03:39:41 +07:00
Rob
9cd5f54e31 [utils] Fix file permissions in write_json_file (closes #12471) (#25122) 2020-05-20 03:21:52 +07:00
9a269547f2 [indavideo] Switch to HTTPS for API request (#25191) 2020-05-20 02:13:06 +07:00
bf097a5077 [redtube] Improve title extraction (#25208) 2020-05-20 02:11:05 +07:00
52c50a10af [vimeo] improve format extraction and sorting(closes #25285) 2020-05-15 15:57:06 +01:00
b334732709 [soundcloud] reduce API playlist page limit(closes #25274) 2020-05-15 14:13:02 +01:00
384bf91f88 [youtube] Add support for yewtu.be (#25226) 2020-05-14 05:54:42 +07:00
fae11394f0 [README.md] flake8 HTTPS URL (#25230) 2020-05-14 05:53:17 +07:00
adc13b0748 [mailru] Fix extraction (closes #24530) (#25239) 2020-05-14 05:51:40 +07:00
327593257c [bbccouk] PEP8 2020-05-14 05:11:42 +07:00
9d8f3a12a6 [spike] fix Bellator mgid extraction(closes #25195) 2020-05-12 20:49:08 +01:00
b002bc433a release 2020.05.08 2020-05-08 18:10:37 +07:00
b74896dad1 [ChangeLog] Actualize
[ci skip]
2020-05-08 18:07:05 +07:00
fa3db38333 [youtube] Improve signature cipher extraction (closes #25188) 2020-05-08 17:42:30 +07:00
30fa5c6087 [iprima] Improve extraction (closes #25138) 2020-05-06 23:20:14 +07:00
6c907eb33f [downloader/http] Request last data block of exact remaining size
Always request last data block of exact size remaining to download if possible not the current block size.
2020-05-05 21:43:39 +07:00
f7b42518dc [downloader/http] Finish downloading once received data length matches expected
Always do this if possible, i.e. if Content-Length or expected length is known, not only in test.
This will save unnecessary last extra loop trying to read 0 bytes.
2020-05-05 21:43:39 +07:00
ce7db64bf1 [uol] fix extraction(closes #22007) 2020-05-05 11:19:40 +01:00
1328305851 [orf] Add support for more radio stations (closes #24938) (#24968) 2020-05-05 06:22:50 +07:00
6c22cee673 [extractor/common] Use compat_cookiejar_Cookie for _set_cookie (closes #23256, closes #24776)
To always ensure cookie name and value are bytestrings on python 2.
2020-05-05 06:00:37 +07:00
6d874fee2a [compat] Introduce compat_cookiejar_Cookie 2020-05-05 05:54:10 +07:00
676723e0da [dailymotion] Fix typo 2020-05-05 05:09:07 +07:00
c380cc28c4 [utils] Improve cookie files support
+ Add support for UTF-8 in cookie files
* Skip malformed cookie file entries instead of crashing (invalid entry len, invalid expires at)
2020-05-05 04:21:25 +07:00
f7f304910d [puhutv] Remove no longer available HTTP formats (closes #25124) 2020-05-04 21:15:19 +07:00
00a41ca4c3 release 2020.05.03 2020-05-03 00:05:05 +07:00
66f32ca0e1 [ChangeLog] Actualize
[ci skip]
2020-05-02 23:59:25 +07:00
6ffc3cf74a [crunchyroll] Fix and improve extraction (closes #25096, closes #25060) 2020-05-02 23:42:51 +07:00
4433bb0245 [extractor/common] Extract multiple JSON-LD entries 2020-05-02 23:40:30 +07:00
e40c758c2a [youtube] Improve player id extraction and add tests 2020-05-02 07:18:08 +07:00
011e75e641 [youtube] Use redirected video id if any (closes #25063) 2020-05-01 00:40:38 +07:00
2468a6fa64 [yahoo] fix GYAO Player extraction and relax title URL regex(closes #24178)(closes #24778) 2020-04-29 14:56:32 +01:00
700265bfcf [tvplay] fix Viafree extraction(closes #15189)(closes #24473)(closes #24789) 2020-04-29 13:38:58 +01:00
c97f5e934f [tenplay] Relax _VALID_URL (closes #25001) 2020-04-26 12:41:33 +07:00
38db9a405a [prosiebensat1] Extract series metadata 2020-04-24 02:56:10 +07:00
2cdfe977d7 [prosiebensat1] Improve extraction and remove 7tv.de support (#24948) 2020-04-24 02:44:13 +07:00
46d0baf941 [options] Clarify doc on --exec command (closes #19087) (#24883) 2020-04-24 02:31:38 +07:00
00eb865b3c [youtube] Fix DRM videos detection (refs #24736) 2020-04-11 23:05:08 +07:00
2f19835726 [thisoldhouse] Improve video id extraction (closes #24549) 2020-04-11 20:07:37 +07:00
533f3e3557 [thisoldhouse] Fix video id extraction (closes #24548)
Added support for:
with of without "www."
and either  ".chorus.build" or ".com"

It now validated correctly on older URL's
```
<iframe src="https://thisoldhouse.chorus.build/videos/zype/5e33baec27d2e50001d5f52f
```
and newer ones
```
<iframe src="https://www.thisoldhouse.com/videos/zype/5e2b70e95216cc0001615120
```
2020-04-11 20:07:32 +07:00
75294a5ed0 [soundcloud] Improve AAC format extraction (closes #19173, closes #24708) 2020-04-10 17:26:03 +07:00
tom
b9e5f87291 [soundcloud] Extract AAC format 2020-04-10 17:25:04 +07:00
6b09401b0b [youtube] Skip broken multifeed videos (closes #24711) 2020-04-09 22:42:43 +07:00
5caf88ccb4 [nova:embed] Fix extraction (closes #24700) 2020-04-09 03:52:29 +07:00
dcc8522fdb [motherless] Fix extraction (closes #24699) 2020-04-09 02:14:49 +07:00
c9595ee780 [twitch:clips] Extend _VALID_URL (closes #24290) (#24642) 2020-04-07 23:21:25 +07:00
91bd3bd019 [tv4] Fix ISM formats extraction (closes #24667) 2020-04-07 22:56:06 +07:00
13b08034b5 [extractor/common] Skip malformed ISM manifest XMLs while extracting ISM formats (#24667) 2020-04-07 22:55:59 +07:00
6a6e1a0cd8 [tele5] Fix extraction (closes #24553) 2020-04-06 02:05:06 +07:00
4e7b5bba5f [mofosex] Add support for generic embeds (closes #24633) 2020-04-06 01:29:58 +07:00
52c4c51556 [youporn] Add support form generic embeds 2020-04-05 20:56:14 +07:00
8fae1a04eb [spankwire] Add support for generic embeds (refs #24633) 2020-04-05 20:42:56 +07:00
d44a707fdd [spankwire] Fix extraction (closes #18924, closes #20648) 2020-04-05 20:42:56 +07:00
049c0486bb release 2020.03.24 2020-03-24 03:14:30 +07:00
30b5121a1c [ChangeLog] Actualize
[ci skip]
2020-03-24 03:12:15 +07:00
b439634f0e [ChangeLog] Actualize
[ci skip]
2020-03-24 03:07:34 +07:00
6e47200b6e [teachable] Update test 2020-03-24 02:57:53 +07:00
38fa761a45 [teachable] Update gns3 domain 2020-03-24 02:57:48 +07:00
08a27407c4 [teachable] Update upskillcourses domain
New version does not use teachable platform any longer
2020-03-24 02:57:44 +07:00
be7dacf9cf [generic] Look for teachable embeds before wistia 2020-03-24 02:57:38 +07:00
4560adc820 [teachable] Extract chapter metadata (closes #24421) 2020-03-24 02:57:32 +07:00
63dce3094b [bilibili] Add support for player.bilibili.com (closes #24402) 2020-03-24 00:24:39 +07:00
b4eb08bb03 [bilibili] Add support for new URL schema with BV ids (closes #24439, closes #24442) 2020-03-24 00:11:39 +07:00
2e20cb3636 [limelight] remove disabled API requests(closes #24255) 2020-03-23 12:57:10 +01:00
a6c5859d6b [soundcloud] fix download url extraction(closes #24394) 2020-03-22 09:24:26 +01:00
c76cdf2382 [cbc:watch] Fix authenticated device token caching (closes #19160) 2020-03-21 01:43:13 +07:00
787c360467 [cbc:watch] Add support for authentication 2020-03-21 01:43:08 +07:00
73453430c1 [hellporno] Fix extraction (closes #24399) 2020-03-21 00:59:48 +07:00
158bc5ac03 [xtube] Fix typo 2020-03-14 22:58:10 +07:00
4568a11802 [xtube] Fix formats extraction (closes #24348) 2020-03-14 22:57:10 +07:00
4cbce88f8b [ndr] Fix extraction (closes #24326) 2020-03-14 04:58:24 +07:00
541fe3eaff [nhk] Update m3u8 URL and use native hls (#24329) 2020-03-14 04:42:40 +07:00
9bfe088594 [nhk] Remove obsolete rtmp formats (closes #24329) 2020-03-14 04:40:11 +07:00
fcaf4d7a06 [nhk] Relax _VALID_URL (#24329) 2020-03-14 04:39:21 +07:00
40b6495d40 Revert "[vimeo] fix showcase password protected video extraction(closes #24224)"
This reverts commit 12ee431676.
2020-03-13 08:59:10 +01:00
f1a8511f7b [utils] Add reference to cookie file format 2020-03-10 04:59:02 +07:00
042b664933 Revert "[utils] Add support for cookies with spaces used instead of tabs"
According to [1] TABs must be used as separators between fields.
Files produces by some tools with spaces as separators are considered
malformed.

1. https://curl.haxx.se/docs/http-cookies.html

This reverts commit cff99c91d1.
2020-03-10 04:53:51 +07:00
68fa15155f release 2020.03.08 2020-03-08 18:27:20 +07:00
434f573046 [ChangeLog] Actualize
[ci skip]
2020-03-08 18:16:17 +07:00
cff99c91d1 [utils] Add support for cookies with spaces used instead of tabs 2020-03-08 18:01:32 +07:00
fa9b8c6628 [pornhub] Add support for pornhubpremium.com (#24288) 2020-03-08 18:00:25 +07:00
ea782aca52 [README.md] Clarify 429 error 2020-03-08 09:17:17 +07:00
43ebf77df3 [youtube] Remove outdated code
Additional get_video_info requests don't seem to provide any extra itags any longer
2020-03-08 08:59:58 +07:00
d332ec725d [youtube] Improve age-gated videos extraction in 429 error conditions (refs #24283) 2020-03-08 05:41:04 +07:00
f93abcf1da [youtube] Improve extraction in 429 error conditions (closes #24283) 2020-03-08 05:09:02 +07:00
0ec9d4e565 [nhk] update API version(closes #24270) 2020-03-06 20:13:28 +01:00
34525a3885 release 2020.03.06 2020-03-06 00:25:43 +07:00
2db9ac228d [ChangeLog] Actualize
[ci skip]
2020-03-06 00:23:14 +07:00
5429d6a9cb [youtube] Fix tests 2020-03-06 00:05:50 +07:00
dc879c5a37 [youtube] Fix age-gated videos support without login (closes #24248) 2020-03-05 23:48:25 +07:00
12ee431676 [vimeo] fix showcase password protected video extraction(closes #24224) 2020-03-03 12:33:57 +01:00
46cc54ca8f [pornhub] Improve title extraction (closes #24184) 2020-03-03 06:23:39 +07:00
1e1c1960aa [peertube] Fix issues and improve extraction (closes #23657) 2020-03-03 03:01:47 +07:00
ac379fa236 [peertube] Improve extraction 2020-03-03 03:01:42 +07:00
jxu
0e30a7b973 [youtube:playlist] Fix tests (closes #23872) (#23885) 2020-03-03 01:46:00 +07:00
3b5399ce0f [servus] Add support for new URL schema (closes #23475, closes #23583, closes #24142) 2020-03-03 01:41:53 +07:00
1c45ff5572 [vimeo] Fix subtitles URLs (#24209) 2020-03-03 01:27:40 +07:00
669625a32c release 2020.03.01 2020-03-01 20:11:32 +07:00
170f5b7c27 [ChangeLog] Actualize
[ci skip]
2020-03-01 20:09:05 +07:00
b274e48d56 [xhamster] Fix extraction (closes #24205) 2020-03-01 20:04:48 +07:00
50d19895a1 [franceculture] Fix extraction (closes #24204) 2020-03-01 19:22:09 +07:00
6d475d01d8 [telecinco] Add support for article opening videos 2020-03-01 03:09:19 +07:00
f8cbd8c963 [telecinco] Fix extraction (refs #24195) 2020-03-01 01:04:51 +07:00
838f051c4b [xtube:user] Fix test 2020-02-29 23:51:56 +07:00
e88b450771 [xtube] Fix metadata extraction (closes #21073, closes #22455) 2020-02-29 23:51:34 +07:00
278355bae4 [zapiks] Fix test 2020-02-29 23:09:13 +07:00
b4cbdbd4b3 [zdf:channel] Fix tests 2020-02-29 23:06:36 +07:00
ea17979d83 [test_subtitles] Remove obsolete test 2020-02-29 22:08:43 +07:00
886d985959 [youjizz] Fix extraction (closes #24181) 2020-02-29 21:58:22 +07:00
7947a1f7db Remove no longer needed compat_str around geturl 2020-02-29 19:19:24 +07:00
fca6dba8b8 [YoutubeDL] Force redirect URL to unicode on python 2 2020-02-29 19:08:44 +07:00
e2f8bf5888 [extractor/common] Convert ISM manifest to unicode before processing on python 2 (#24152) 2020-02-29 17:29:30 +07:00
b76f0e58f7 [options] Remove duplicate short option -v for --version (#24162) 2020-02-29 16:33:09 +07:00
bee6451fe8 [pornhd] Fix extraction (closes #24128) 2020-02-24 04:47:56 +07:00
00d798b7c2 [teachable] Add support for multiple videos per lecture (closes #24101) 2020-02-23 06:49:45 +07:00
fda6d237a5 [wistia] Add support for multiple generic embeds (closes #8347, closes #11385) 2020-02-23 06:47:11 +07:00
5d9f6cbc5a [imdb] Fix extraction (closes #23443) 2020-02-23 04:33:29 +07:00
97c822b3d5 [tv2dk:bornholm:play] Fix extraction (#24076) 2020-02-19 01:02:05 +07:00
117ba9e9df release 2020.02.16 2020-02-16 22:43:42 +07:00
0d718db623 [ChangeLog] Actualize
[ci skip]
2020-02-16 22:40:44 +07:00
7bf27721d6 [npr] Add support for streams (closes #24042) 2020-02-15 05:35:55 +07:00
f6052ec923 [24video] Add support for porn.24video.net (closes #23779, closes #23784) 2020-02-15 03:49:29 +07:00
4e9e1e240d [test_YoutubeDL] Add tests for #10591 (closes #23873) 2020-02-15 03:37:31 +07:00
e0abaab293 [test_YoutubeDL] Fix get_ids 2020-02-15 03:37:25 +07:00
jxu
de1121d749 [YoutubeDL] Fix playlist entry indexing with --playlist-items (closes #10591, closes #10622) 2020-02-15 03:36:53 +07:00
293c9f0186 [jpopsuki] Remove extractor (closes #23858) 2020-02-15 02:23:29 +07:00
06f1de2daf [nova] Improve extraction (refs #23690) 2020-02-15 02:16:26 +07:00
b68a6e32fb [nova:embed] Improve (closes #23690) 2020-02-15 02:00:58 +07:00
8cd809fb3d [nova:embed] Fix extraction (closes #23672) 2020-02-15 02:00:52 +07:00
d6aa1db7ed [abc:iview] Support 720p (#22907) (#22921) 2020-02-13 14:52:00 +01:00
f377edec06 [nytimes] improve format sorting(closes #24010) 2020-02-10 09:43:20 +01:00
bfe2b8cf2a [update] Fix updating via symlinks (closes #23991) 2020-02-08 19:46:58 +07:00
82fea5b42e [compat] Introduce compat_realpath (refs #23991) 2020-02-08 19:36:55 +07:00
fffc618c51 [toggle] Add support for mewatch.sg (closes #23895) (#23930) 2020-02-05 22:41:56 +07:00
705b1cda99 [thisoldhouse] fix extraction(closes #23951) 2020-02-03 13:20:36 +01:00
7d55b62ff2 [popcorntimes] Add extractor (closes #23949) 2020-02-03 06:05:56 +07:00
0d006fac5c [sportdeutschland] Update to new sportdeutschland API
They switched to SSL, but under a different host AND path...
Remove the old test cases because these videos have become unavailable.
2020-02-01 23:35:55 +01:00
00de61a98f [twitch:stream] Lowercase channel id for stream request (closes #23917) 2020-02-01 00:32:25 +07:00
d95a1cc98e [tv5mondeplus] Fix extraction (closes #23907, closes #23911) 2020-01-31 04:58:36 +07:00
4935749730 [tva] Relax _VALID_URL (closes #23903) 2020-01-31 03:49:16 +07:00
51c7f40c83 [vimeo] fix album extraction(closes #23864) 2020-01-27 23:37:29 +01:00
4877ffc0e9 [viewlift] improve extraction
- fix extraction(closes #23851)
- add add support for authentication
- add support for more domains
2020-01-27 15:41:21 +01:00
8e4d3f83ce [svt] fix series extraction(closes #22297) 2020-01-26 16:17:51 +01:00
43e7994749 [svt] fix article extraction(closes #22897)(closes #22919) 2020-01-26 14:16:59 +01:00
2a5c26c980 [soundcloud] imporve private playlist/set tracks extraction
https://github.com/ytdl-org/youtube-dl/issues/3707#issuecomment-577873539
2020-01-23 23:24:37 +01:00
76dbe4df5f release 2020.01.24 2020-01-24 04:16:05 +07:00
bffdedfabd [ChangeLog] Actualize
[ci skip]
2020-01-24 04:14:08 +07:00
c3cfea9068 [youtube] Fix sigfunc name extraction (closes #23819) 2020-01-24 04:09:10 +07:00
22cb94902f [stretchinternet] fix extraction(closes #4319) 2020-01-19 21:20:56 +01:00
be96f9924f [voicerepublic] fix extraction 2020-01-19 20:15:02 +01:00
9cf30dc017 [azmedien] fix extraction(closes #23783) 2020-01-19 19:30:48 +01:00
f4a18db748 [ard] add a missing condition 2020-01-19 18:28:24 +01:00
PB
fd032450f0 [businessinsider] Fix jwplatform id extraction (closes #22929) (#22954) 2020-01-18 22:47:50 +07:00
a4b2769451 [24video] Add support for 24video.vip (closes #23753) 2020-01-18 15:05:45 +07:00
d9a2f86791 [ivi:compilation] Fix entries extraction (closes #23770) 2020-01-18 14:46:38 +07:00
c968f738df [ard] improve extraction(closes #23761)
- simplify extraction
- extract age limit and series
- bypass geo-restriction
2020-01-17 14:23:24 +01:00
48ff5590c1 [nbc] add support for nbc multi network URLs(closes #23049) 2020-01-16 15:37:16 +01:00
2c482bff7c [americastestkitchen] fix extraction 2020-01-15 14:18:04 +01:00
a9866c0366 [zype] improve extraction
- extract subtitles(closes #21258)
- support URLs with alternative keys/tokens(#21258)
- extract more metadata
2020-01-15 14:18:04 +01:00
90ea83c64d [orf:tvthek] Improve geo restricted videos detection (closes #23741) 2020-01-15 04:32:05 +07:00
e4e5fa6e3c [soundcloud] Restore previews extraction (closes #23739) 2020-01-15 04:13:10 +07:00
e8cf0dbdd8 release 2020.01.15 2020-01-15 01:37:29 +07:00
d7c55f226d [ChangeLog] Actualize
[ci skip]
2020-01-15 01:34:01 +07:00
bfdc8340c9 [yourporn] Fix extraction (closes #21645, closes #22255, closes #23459) 2020-01-15 01:28:17 +07:00
14bb191634 [travis] Add flake8 job (#23720) 2020-01-15 01:09:08 +07:00
628e5bc0b7 [canvas] Add support for new API endpoint and update tests (closes #17680, closes #18629) 2020-01-14 23:53:59 +07:00
3fc56635b7 [ndr:base:embed] Improve thumbnails extraction (closes #23731) 2020-01-14 21:46:56 +07:00
bd2c211fcc [vodplatform] add support for embed.kwikmotion.com domain 2020-01-12 17:34:57 +01:00
10a5091e58 [twitter] add support for promo_video_website cards(closes #23711) 2020-01-12 12:01:59 +01:00
aca2fd222f [orf:radio] Clean description and improve extraction 2020-01-11 02:18:36 +07:00
9ba179c1fa [orf:fm4] Fix extraction (#23599) 2020-01-11 01:51:15 +07:00
3fdf573148 [safari] Fix kaltura session extraction (closes #23679) (#23670) 2020-01-11 01:34:26 +07:00
d4e0cd69ef [lego] fix extraction and extract subtitle(closes #23687) 2020-01-10 05:06:45 +01:00
483b858d49 [cloudflarestream] import embed URL extraction 2020-01-08 23:07:41 +01:00
a71c1d1a5a [cloudflarestream] improve extraction
- add support for bytehighway.net domain
- add support for signed URLs
- extract thumbnail
2020-01-08 22:42:53 +01:00
838171630d [naver] improve metadata extraction 2020-01-08 12:55:33 +01:00
c88debff5d [naver] improve extraction
- improve geo-restriction handling
- extract automatic captions
- extract uploader metadata
- extract VLive HLS formats
2020-01-08 10:59:56 +01:00
3cb05b86de [pandatv] Remove extractor (#23630) 2020-01-07 21:11:03 +07:00
b2771a2853 [dctp] fix format extraction(closes #23656) 2020-01-07 13:03:32 +01:00
7bac77413d [scrippsnetworks] correct test case URL 2020-01-06 14:30:02 +01:00
0264903574 [scrippsnetworks] add support for www.discovery.com videos 2020-01-06 14:25:54 +01:00
2f7aa680b7 [discovery] fix anonymous token extraction(closes #23650) 2020-01-06 14:25:54 +01:00
0d2306d02b [nrktv:seriebase] Fix extraction (closes #23625) (#23537) 2020-01-06 06:34:36 +07:00
233826f68f [wistia] improve format extraction and extract subtitles(closes #22590) 2020-01-05 21:09:37 +01:00
259ad38173 [devscripts/create-github-release] Remove unused import 2020-01-06 01:26:22 +07:00
44b434e4e3 [vice] improve extraction(closes #23631) 2020-01-05 16:33:21 +01:00
484637a9cc [redtube] Detect private videos (#23518) 2020-01-02 22:45:42 +07:00
ca069f6881 release 2020.01.01 2020-01-01 05:24:58 +07:00
0d5c415e1f [devscripts/create-github-release] Switch to using PAT for authentication
Basic authentication will be deprecated soon
2020-01-01 05:20:48 +07:00
d6bf9cbd46 [ChangeLog] Actualize
[ci skip]
2020-01-01 04:13:32 +07:00
de7aade2f8 [soundcloud] fix client id extraction for non fatal requests 2019-12-31 21:31:22 +01:00
2d30b92e11 [brightcove] invalidate policy key cache on failing requests 2019-12-31 19:49:01 +01:00
0164cd5dac [pornhub] Improve locked videos detection (closes #22449, closes #22780) 2019-12-31 23:43:43 +07:00
f41347260c [pornhub] Fix extraction and add support for m3u8 formats (closes #22749, closes #23082) 2019-12-31 23:29:06 +07:00
0606808746 [brightcove] update policy key on failing requests 2019-12-31 16:44:30 +01:00
0a02732b56 [spankbang] Improve removed video detection (#23423) 2019-12-31 22:18:01 +07:00
2b845c4086 [spankbang] Fix extraction (closes #23307, closes #23423, closes #23444) 2019-12-31 22:16:39 +07:00
3bed621750 [soundcloud] automatically update client id on failing requests 2019-12-31 09:49:29 +01:00
0c15a56f1c [prosiebensat1] improve geo restriction handling(closes #23571) 2019-12-30 22:31:11 +01:00
75ef77c1b1 [brightcove] cache brightcove player policy keys 2019-12-29 19:31:17 +01:00
cb7e053e0a [extractors] add missing import for ScrippsNetworksIE 2019-12-29 19:31:17 +01:00
941e359e95 [teachable] Fail with error message if no video URL found 2019-12-27 00:26:12 +07:00
f8a12427a9 [teachable] Improve locked lessons detection (#23528) 2019-12-27 00:18:37 +07:00
7ea55819ac [scrippsnetworks] Add new extractor(closes #19857)(closes #22981) 2019-12-26 15:25:04 +01:00
18ff573e50 [mitele] fix extraction(closes #21354)(closes #23456) 2019-12-25 20:02:31 +01:00
d1b2722095 [soundcloud] Update client id (closes #23516) 2019-12-25 22:39:50 +07:00
278be57be2 [mailru] Relax _VALID_URLs (#23509) 2019-12-25 04:28:34 +07:00
80e43af5bf release 2019.12.25 2019-12-25 01:16:49 +07:00
b1a92520a3 [ChangeLog] Actualize
[ci skip]
2019-12-25 00:52:11 +07:00
9b6e72fd06 [mediaset] Fix parse formats (closes #23508) 2019-12-24 23:51:08 +07:00
2dbc0967f2 [ChangeLog] Actualize
[ci skip]
2019-12-16 00:40:34 +07:00
fab01080f4 [tv2dk:bornholm:play] Add extractor (closes #23291) 2019-12-16 00:08:18 +07:00
42db58ec73 [utils] Improve str_to_int 2019-12-15 23:15:24 +07:00
73d8f3a634 [slideslive] add support for url and vimeo service names(closes #23414) 2019-12-14 21:35:31 +01:00
b33a05d221 [slideslive] fix extraction(closes #23413) 2019-12-14 19:29:04 +01:00
232ed8e6e0 [twitch] fix clip extraction(closes #23375) 2019-12-13 11:00:31 +01:00
cf80ff186e [soundcloud] add support for token protected embeds(#18954) 2019-12-09 14:38:12 +01:00
0e6ec3caf6 [vk] improve extraction
- fix User Videos extraction(closes #23356)
- extract all videos for lists with more than 1000 videos(#23356)
- add support for video albums(closes #14327)(closes #14492)
2019-12-09 09:13:02 +01:00
d686cab084 [kontrtube] remove extractor 2019-12-08 12:38:21 +01:00
9d4424afaa [videopremium] remove extractor 2019-12-08 11:54:16 +01:00
ce709fcb00 [musicplayon] remove extractor(closes #9225) 2019-12-07 20:17:30 +01:00
6633103f8e [ufctv] add support for ufcfightpass.imgdge.com and ufcfightpass.imggaming.com domains(closes #23343) 2019-12-07 19:23:19 +01:00
1d31b7ca04 [twitch] extract m3u8 formats frame rate(closes #23333) 2019-12-06 15:34:35 +01:00
4067a23270 [ufctv] add support for more domains and remove compatibility code(closes #23332) 2019-12-06 11:04:12 +01:00
7d53fa475a [imggaming] add support for playlists and extract subtitles 2019-12-04 20:56:23 +01:00
3ae878605d [ufctv] fix extraction and add support for UFC Arabia(closes #23312) 2019-12-04 17:20:53 +01:00
22974a3782 [yahoo] correct gyao brightcove player id(closes #23303) 2019-12-03 21:13:44 +01:00
63fe44eb4d [vzaar] update test 2019-12-03 12:31:16 +01:00
c712b16dc4 [vzaar] override AES decryption key URL(closes #17521) 2019-12-03 12:23:08 +01:00
6797de75e0 [vzaar] add support for AES HLS manifests(closes #17521)(closes #23299) 2019-12-03 11:37:30 +01:00
12cc89122d [nrl] fix extraction 2019-11-30 23:50:28 +01:00
3765284476 [teachingchannel] fix extraction 2019-11-30 23:49:45 +01:00
ddfe50195b [nintendo] fix extraction and partially add support for Nintendo Direct videos(#4592) 2019-11-30 23:48:26 +01:00
1ed2c4b378 [ooyala] add better fallback values for domain and streams variables 2019-11-30 23:21:13 +01:00
66b4872747 [youtube] add support youtubekids.com(closes #23272) 2019-11-30 17:51:34 +01:00
0b25af9bf5 [tv2] detect DRM protection 2019-11-30 15:50:17 +01:00
8d3a3a9901 [tv2] add support for mtv.fi and fix tv2.no article extraction(closes #10543) 2019-11-30 15:26:12 +01:00
c0b1e01330 [msn] improve extraction
- add support for YouTube and NBCSports embeds
- add support for aricles with multiple videos
- improve AOL embed support
- improve format extraction
2019-11-29 17:39:18 +01:00
88a7a9089a [abcotvs] relax _VALID_URL regex and improve metadata extraction(closes #18014) 2019-11-29 17:39:18 +01:00
a15adbe461 [channel9] reduce response size and update tests 2019-11-29 17:39:18 +01:00
7f641d2c7a [adobetv] improve extaction
- use OnDemandPagedList for list extractors
- reduce show extraction requests
- extract original video format and subtitles
- add support for adobe tv embeds
2019-11-29 17:39:18 +01:00
348c6bf1c1 [utils] handle int values passed to str_to_int 2019-11-29 17:39:18 +01:00
b568561eba release 2019.11.28 2019-11-28 23:25:25 +07:00
e3f00f139f [ChangeLog] Actualize
[ci skip]
2019-11-28 23:09:48 +07:00
681ac7c92a [vimeo] improve extraction
- fix review extraction
- fix ondemand extraction
- make password protected player case as an expected error(closes #22896)
- simplify channel based extractors code
2019-11-27 13:57:30 +01:00
6471d0d3b8 [openload] remove OpenLoad related extractors(closes #11999)(closes #15406) 2019-11-26 23:57:37 +01:00
5ef62fc4ce [dailymotion] improve extraction
- extract http formats included in m3u8 manifest
- fix user extraction(closes #3553)(closes #21415)
- add suport for User Authentication(closes #11491)
- fix password protected videos extraction(closes #23176)
- respect age limit option and family filter cookie value(closes #18437)
- handle video url playlist query param
- report alowed countries for geo-restricted videos
2019-11-26 22:18:21 +01:00
df65a4a1ed [corus] improve extraction
- add support for Series Plus, W Network, YTV, ABC Spark, disneychannel.com
  and disneylachaine.ca(closes #20861)
- add support for self hosted videos(closes #22075)
- detect DRM protection(closes #14910)(closes #9164)
2019-11-26 22:18:21 +01:00
edc2a1f68b [vivo] Fix extraction (closes #22328, closes #22279) 2019-11-27 02:28:06 +07:00
1ced222120 [utils] Add generic caesar cipher and rot47 2019-11-27 02:26:42 +07:00
6ddd4bf6ac [bitchute] Extract upload date (closes #22990) (#23193) 2019-11-27 00:20:39 +07:00
9d30c2132a [utils] Handle rd-suffixed day parts in unified_strdate (#23199) 2019-11-27 00:08:37 +07:00
cf3c9eafad [soundcloud] Update client id (closes #23214) 2019-11-27 00:03:51 +07:00
0de9fd24dc release 2019.11.22 2019-11-22 01:24:27 +07:00
fb8dfc5a27 [ChangeLog] Actualize
[ci skip]
2019-11-22 01:21:00 +07:00
80a51fc2ef [ivi] Skip s353 for bundled exe
See https://github.com/Legrandin/pycryptodome/issues/228
2019-11-22 01:10:24 +07:00
f8015c1574 [ivi] Fix python 3.4 support 2019-11-21 23:38:39 +07:00
25d3f770e6 [ivi] Ask for pycryptodomex instead of pycryptodome
See discussion at 1bba88efc7 (r35982110)
2019-11-21 23:22:59 +07:00
f0f6a7e73f [chaturbate] Fix extraction (closes #23010, closes #23012) 2019-11-21 23:21:03 +07:00
76d9eca43d [ivi] fallback to old extraction method for unknown error codes 2019-11-19 20:16:31 +01:00
f9c4a45210 [ntvru] add support for non relative file URLs(closes #23140) 2019-11-18 21:40:53 +01:00
7e70620a34 [vk] fix wall audio thumbnails extraction(closes #23135) 2019-11-18 12:51:25 +01:00
9e4e864639 [ivi] improve error detection 2019-11-16 01:51:48 +01:00
6c79785bb0 [travis] Add python 3.8 build 2019-11-16 07:47:23 +07:00
7360c06fac [extractor/common] Add data, headers and query to all major extract methods preserving standard order for potential future use 2019-11-16 05:55:54 +07:00
1bba88efc7 [ivi] sign content request only when pycryptodome is available 2019-11-15 23:46:31 +01:00
656c20010f [ivi] fix format extraction(closes #21991) 2019-11-15 21:17:47 +01:00
8b1a30c993 [comcarcoff] remove extractor 2019-11-14 06:39:21 +01:00
5709d661a2 [drtv] Add support for new URL schema (closes #23059) 2019-11-14 01:45:04 +07:00
eb22d1b557 [nexx] Add support for Multi Player JS Setup(closes #23052) 2019-11-13 19:09:32 +01:00
48970d5cc8 [teamcoco] add support for new videos(closes #23054) 2019-11-12 10:51:54 +01:00
2e9ad59a4d [soundcloud] check if the soundtrack has downloads left(closes #23045) 2019-11-11 09:53:04 +01:00
433e071058 [facebook] fix posts video data extraction(closes #22473) 2019-11-10 17:02:47 +01:00
9e46d1f8aa [addanime] remove extractor 2019-11-09 17:15:15 +01:00
88b87b08b1 [minhateca] remove extractor 2019-11-09 17:01:21 +01:00
20baa17c01 [daisuki] remove extractor 2019-11-09 16:00:12 +01:00
8fbf5d2f87 [seeker] remove Revision3 extractors and fix extraction 2019-11-09 13:14:23 +01:00
f81dd65ba2 [extractor/common] clean jwplayer description HTML tags 2019-11-09 13:11:59 +01:00
ce112a8c19 [twitch] fix video comments URL(#18593)(closes #15828) 2019-11-09 11:01:07 +01:00
18ca61c5e1 [twitter] improve extraction
- add support for generic embeds(closes #22168)
- always extract http formats for native videos(closes #14934)
- add support for Twitter Broadcasts(closes #21369)
- extract more metadata
- improve VMap format extraction
- unify extraction code for both twitter statuses and cards
2019-11-09 09:23:20 +01:00
0b16b3c2d3 [twitch] add support for Clip embed URLs 2019-11-09 09:22:24 +01:00
d4f53af482 [lnkgo] fix extraction(closes #16834) 2019-11-06 23:14:26 +01:00
5d92b407e0 [mixcloud] improve extraction
- improve metadata extraction(closes #11721)
- fix playlist extraction(closes #22378)
- fix user mixes extraction(closes #15197)(closes #17865)
2019-11-06 20:41:49 +01:00
55adb63e54 [kinja] add support for Kinja embeds
closes #5756
closes #11282
closes #22237
closes #22384
2019-11-06 19:56:10 +01:00
d64ec1242e [onionstudios] fix extraction 2019-11-06 10:44:19 +01:00
3ec86619e3 [common] initialize headers param with empty dict 2019-11-06 07:18:29 +01:00
57033e35e5 [common] fix typo 2019-11-05 23:41:57 +01:00
d7def23d05 [hotstar] pass Referer header to format requests(closes #22836) 2019-11-05 23:08:42 +01:00
b6139cb0c3 [common] pass headers to _extract_(m3u8|mpd)_formats methods 2019-11-05 22:56:25 +01:00
2318629b2b [dplay] minimize response size 2019-11-05 14:04:50 +01:00
b77c3949e8 [patreon] minimize reponse size and extract uploader_id and filesize 2019-11-05 14:04:17 +01:00
e9b95167af [roosterteeth] fix login request(closes #16094)(closes #22689) 2019-11-05 10:06:02 +01:00
ea07412ebf release 2019.11.05 2019-11-05 05:32:56 +07:00
1a4e4b0bfe [ChangeLog] Actualize
[ci skip]
2019-11-05 05:31:40 +07:00
20218040db [scte] Add extractor (closes #22975) 2019-11-05 05:21:16 +07:00
c69e71733d [msn] add support for Vidible and AOL embeds(closes #22195)(closes #22227) 2019-11-04 22:21:00 +01:00
3e49083604 [myspass] fix video URL extraction and improve metadata extraction(closes #22448) 2019-11-04 20:05:27 +01:00
2349255abd [jamendo] restore track url modification 2019-11-04 15:51:44 +01:00
e452345fc5 [jamendo] improve extraction
- fix album extraction(closes #18564)
- improve metadata extraction(closes #18565)(closes #21379)
2019-11-04 15:43:52 +01:00
bf45295c53 [mediaset] relax URL guid matching(closes #18352) 2019-11-04 11:13:14 +01:00
ef382405c6 [mediaset] extract unprotected M3U and MPD manifests(closes #17204) 2019-11-04 02:02:29 +01:00
a6e6673e82 [README.md] Also read permission to the binary in how to update section (#22903) 2019-11-04 04:23:27 +07:00
564275e26f [telegraaf] fix extraction 2019-11-03 22:04:03 +01:00
726e8eef59 [bellmedia] add support for marilyn.ca videos(#22193) 2019-11-02 22:33:51 +01:00
e54924c46f [stv] fix extraction(closes #22928) 2019-11-02 18:13:31 +01:00
5e36b63486 [iconosquare] remove extractor 2019-11-02 13:25:39 +01:00
9249c50c18 [keek] remove extractor 2019-11-02 13:09:44 +01:00
79b35e7c15 [gameone] Remove extractor (#21778) 2019-11-02 11:32:49 +00:00
836bfcb54e [flipagram] remove extractor 2019-11-02 11:08:51 +01:00
4c95fcf9e8 [bambuser] remove extractor
https://web.archive.org/web/20190808014227/https://go.bambuser.com/shutdown-announcement
2019-11-01 21:16:47 +01:00
152f22920d [wistia] reduce embed extraction false positives and support inline embeds(closes #22931) 2019-11-01 17:44:34 +01:00
20cc7c082b [go90] remove extractor 2019-11-01 16:36:35 +01:00
e987ce4bda [kakao] remove raw request and extract format total bitrate 2019-11-01 12:40:41 +01:00
d439989215 [daum] fix VOD and Clip extracton(closes #15015) 2019-11-01 11:43:18 +01:00
274bf5e4c5 [kakao] improve extraction
- support embed URLs
- support Kakao Legacy vid based embed URLs
- only extract fields used for extraction
- strip description and extract tags
2019-11-01 11:37:41 +01:00
e993f1a095 [mixcloud] fix cloudcast data extraction(closes #22821) 2019-10-31 08:13:10 +01:00
3cf70bf159 [yahoo] make cbs URL suffix part of the media alias 2019-10-31 07:44:21 +01:00
237513e801 [yahoo] restore support for cbs suffixed URLs 2019-10-31 07:38:53 +01:00
8040a0d35e [yahoo] fix typo 2019-10-30 23:52:09 +01:00
45f4a43389 [yahoo] improve extraction
- add support for live streams(closes #3597)(closes #3779)(closes #22178)
- bypass cookie consent page for european domains(closes #16948)(closes #22576)
- add generic support for embeds(closes #20332)
2019-10-30 23:24:49 +01:00
9a621ddc3a [tv2] Fix and improve extraction (closes #22787) 2019-10-30 02:21:52 +07:00
c56b2ac43c [tv2dk] Add extractor 2019-10-30 02:21:03 +07:00
8989349e6d [onet] improve extraction
- add support for onet100.vod.pl domain
- extract m3u8 formats
- correct audio only format info
2019-10-29 09:50:01 +01:00
7455832f31 [fox9] fix extraction 2019-10-29 09:50:00 +01:00
c4bd9cb7bb release 2019.10.29 2019-10-29 06:12:33 +07:00
cae0bbc538 [ChangeLog] Actualize
[ci skip]
2019-10-29 06:11:09 +07:00
53896ca5be [utils] Actualize major IPv4 address blocks per country 2019-10-29 06:10:20 +07:00
0d7392e68b [ChangeLog] Actualize
[ci skip]
2019-10-29 05:54:32 +07:00
aef9f87ea4 [go] Improve and beautify _VALID_URL 2019-10-29 05:52:15 +07:00
dd90a21c28 [go] Add support for abc.com and freeform.com (closes #22823, closes #22864) 2019-10-29 05:49:36 +07:00
01358b9fc1 [extractors] add import for MTVJapanIE 2019-10-28 23:34:31 +01:00
3cdcebf547 [mtv] add support for mtvjapan.com 2019-10-28 23:31:14 +01:00
cfabc50598 [mtv] fix extraction for mtv.de (closes #22113) 2019-10-28 22:55:01 +01:00
0086726e86 [videodetective] fix extraction 2019-10-28 19:48:34 +01:00
83e49259bf [internetvideoarchive] fix extraction 2019-10-28 19:47:27 +01:00
895e5c03db [nbcnews] fix extraction
closes #12569
closes #12576
closes #21703
closes #21923
2019-10-28 19:31:20 +01:00
702984eca9 [hark] remove extractor 2019-10-28 17:49:05 +01:00
b3c2fa6dad [tutv] remove extractor 2019-10-28 17:42:33 +01:00
831b732da1 [learnr] remove extractor 2019-10-28 17:41:17 +01:00
3e252cca0e [macgamestore] remove extractor
Covered by generic extractor
2019-10-28 17:39:01 +01:00
0f9d53566a [la7] update Kaltura service URL(closes #22358) 2019-10-28 15:17:06 +01:00
80c2126e80 [thesun] fix extraction(closes #16966) 2019-10-28 13:32:35 +01:00
71fa0b04f9 [makertv] remove extractor 2019-10-28 13:30:30 +01:00
dd90451f0f [tenplay] Add new extractor(closes #21446) 2019-10-27 22:02:46 +01:00
548c395716 [soundcloud] improve extraction
- improve format extraction(closes #22123)
- extract uploader_id and uploader_url(closes #21916)
- extract all known thumbnails(closes #19071)(closes #20659)
- fix extration for private playlists(closes #20976)
- add support for playlist embeds(#20976)
- skip preview formats(closes #22806)
2019-10-27 17:52:46 +01:00
0b98f3a751 [dplay] improve extraction
- add support for dplay.fi, dplay.jp and es.dplay.com(closes #16969)
- fix it.dplay.com extraction(closes #22826)
- update tests
- extract creator, tags and thumbnails
- handle playback API call errors
2019-10-26 14:58:29 +01:00
235dbb434b [discoverynetworks] add support for dplay.co.uk 2019-10-26 14:57:42 +01:00
42cd0824b3 [vk] remove assert statement 2019-10-26 00:06:05 +01:00
3c989818e7 [vk] improve extraction
- add support for Odnoklassniki embeds
- update tests
- extract more video from user lists(closes #4470)
- fix wall post audio extraction(closes #18332)
- improve error detection(closes #22568)
2019-10-25 19:35:07 +01:00
416c3ca7f5 [odnoklassniki] add support for Schemeless embed extraction 2019-10-25 19:27:28 +01:00
162bcc68dc [puhutv] improve extraction
- fix subtitles extraction
- transform HLS URLs to http URLs
- improve metadata extraction
2019-10-24 12:53:33 +01:00
07154c7930 [facebook] extract subtitles(closes #22777) 2019-10-22 17:59:14 +01:00
0c2d10d225 [globo] handle alternative hash signing method 2019-10-22 17:59:14 +01:00
820215f0e3 release 2019.10.22 2019-10-22 00:09:02 +07:00
b4818e3c7a [ChangeLog] Actualize
[ci skip]
2019-10-22 00:06:48 +07:00
2297c0d7d9 [facebook] Bypass download rate limits (closes #21018) 2019-10-19 23:56:36 +07:00
824fa51165 [utils] Improve subtitles_filename (closes #22753) 2019-10-18 04:03:53 +07:00
34e3885bc9 [viewster->contv] remove viewster extractor and add support for contv.com 2019-10-17 15:55:44 +01:00
59296bae7e [xfileshare] clean extractor
- update the list of domains
- add support for aa-encoded video data
- improve jwplayer format extraction
- add support for Clappr sources

closes #17032
closes #17906
closes #18237
closes #18239
2019-10-17 13:26:45 +01:00
755541a4c8 [mangomolo] fix video format extraction and add support for player URLs 2019-10-17 13:21:44 +01:00
86f63633c8 [audioboom] improve metadata extraction 2019-10-17 13:20:16 +01:00
0001157594 [atresplayer] Add coding cookie 2019-10-16 23:57:40 +01:00
bc48773ed4 [twitch] update VOD URL matching (closes #22395) (#22727) 2019-10-16 15:13:35 +00:00
d07866f13e [mit] Remove support for video.mit.edu(closes #22403) 2019-10-16 15:45:45 +01:00
2b115b9460 [servingsys] Remove extractor(closes #22639) 2019-10-16 15:41:58 +01:00
e29e96a9f5 [dumpert] fix extraction(closes #22428)(closes #22564) 2019-10-16 15:06:48 +01:00
6d394a66f5 [atresplayer] fix extraction(closes #16277)(closes #16716) 2019-10-16 12:04:52 +01:00
7815d6b743 release 2019.10.16 2019-10-16 03:26:47 +07:00
173190f5e3 [ChangeLog] Actualize
[ci skip]
2019-10-16 03:25:13 +07:00
974311b5aa [vimeo] improve album videos id extraction(closes #22599) 2019-10-15 21:01:59 +01:00
30eb05cb41 [globo] extract subtitles(closes #22713) 2019-10-15 19:54:53 +01:00
2af01c0293 [bokecc] improve player params extraction(closes #22638) 2019-10-15 15:18:51 +01:00
7e05df71b7 [nexx] handle result list(closes #22666) 2019-10-15 00:10:22 +01:00
a1ee23e98f [vimeo] fix VHX embed extraction 2019-10-14 18:37:35 +01:00
311ee45731 [nbc] switch to graphql api(closes #18581)(closes #22693)(closes #22701) 2019-10-14 18:36:25 +01:00
c317b6163b [vessel] remove extractor 2019-10-10 00:01:51 +01:00
2765c47a8c [promptfile] Remove extractor (closes #6239) 2019-10-10 03:40:01 +07:00
07b50f616e [kaltura] Fix service URL extraction (closes #22658) 2019-10-10 00:24:03 +07:00
1907f06e7b [kaltura] Fix embed info strip (refs #22658) 2019-10-10 00:11:41 +07:00
d4bb825b83 [globo] fix format extraction(closes #20319) 2019-10-09 11:08:28 +01:00
560d3b7d7c [redtube] Improve metadata extraction (closes #22492, closes #22615) 2019-10-05 22:04:49 +07:00
4bf568d36c [pornhub:uservideos:upload] Fix extraction (closes #22619) 2019-10-05 21:43:31 +07:00
05446d483d [telequebec:squat] Add support for squat.telequebec.tv (closes #18503) 2019-10-04 20:17:18 +07:00
3a37f2c3be [wimp] Remove extractor (closes #22088) (#22091) 2019-10-04 19:48:20 +07:00
0b87beefe6 [gfycat] Extend _VALID_URL (#22225) 2019-10-04 19:27:58 +07:00
fd4db1ebc2 [chaturbate] Extend _VALID_URL (#22309) 2019-10-04 19:22:01 +07:00
b64045cd2a [peertube] Update instances (#22414) 2019-10-04 19:17:16 +07:00
c2915de82e [telequebec] Add support for coucou.telequebec.tv (#22482) 2019-10-04 19:14:31 +07:00
4e72d02f39 [xvideos] Extend _VALID_URL (#22471) 2019-10-04 19:05:35 +07:00
76e510b92c [youtube] Remove support for invidious.enkirton.net (#22543) 2019-10-04 19:01:03 +07:00
9679a62a28 [openload] Add support for oload.monster (#22592) 2019-10-04 18:57:51 +07:00
ca20b13048 [nrktv:seriebase] Fix extraction (#22596) 2019-10-04 18:57:18 +07:00
894b3826f5 [youtube] Add support for yt.lelux.fi (#22597) 2019-10-04 18:52:15 +07:00
aaf9d904aa [orf:tvthek] Make manifest requests non fatal (refs #22578) 2019-10-03 00:55:46 +07:00
25e911a968 [extractor/common] Make _is_valid_url more relaxed 2019-10-03 00:53:07 +07:00
74bc299453 [teachable] Skip login when already logged in (closes #22572) 2019-10-02 02:03:22 +07:00
2906631e12 [viewlift] Fix URL matching 2019-10-01 23:18:11 +07:00
326ae4ff96 [viewlift] Improve extraction (closes #22545) 2019-09-29 23:03:39 +07:00
72fd4d0c6a [nonktube] Fix extraction (closes #22544) 2019-09-29 21:57:08 +07:00
f4b865c613 release 2019.09.28 2019-09-28 00:30:30 +07:00
412f44f4b3 [ChangeLog] Actualize
[ci skip]
2019-09-28 00:23:25 +07:00
6483fbd336 [vk] Fix extraction (closes #22522) 2019-09-28 00:04:52 +07:00
8130ac42e5 [openload] PEP 8 2019-09-26 23:15:06 +07:00
cb3e4a2947 [heise] Fix kaltura embeds extraction (closes #22514) 2019-09-26 23:11:02 +07:00
2a88a0c44d [ted] check for resources validity and extract subtitled downloads(closes #22513) 2019-09-26 11:44:57 +01:00
33c1c7d80f [youtube] Add support for owxfohz4kjyv25fvlqilyxast7inivgiktls3th44jhk3ej3i7ya.b32.i2p (#22292) 2019-09-25 02:43:34 +07:00
21d3c21e62 [nhk] Add support for clips 2019-09-25 02:39:25 +07:00
a373befa25 [nhk] fix video extraction(closes #22249)(closes #22353) 2019-09-24 20:24:17 +01:00
df63cafe49 [byutv] Fix extraction (refs #22070)
Downloading of new videos does not work due to DRM
2019-09-25 02:16:25 +07:00
d06daf23da [YoutubeDL] Honour all --get-* options with --flat-playlist (closes #22493) 2019-09-25 02:10:37 +07:00
8e9fdcbe27 [openload] Add support for oload.online (#22304) 2019-09-24 23:56:12 +07:00
666d808e70 [youtube] Add support for invidious.drycat.fr (#22451) 2019-09-24 23:16:46 +07:00
7d327fea5b [jwplatfom] do not match video URLs(#20596) (#22148) 2019-09-23 19:44:00 +00:00
4e3f1f0469 [youtube:playlist] Unescape playlist uploader (closes #22483) 2019-09-23 00:20:52 +07:00
4bc15a68d1 [bilibili] add support audio albums and songs(closes #21094) 2019-09-22 17:14:18 +01:00
edb2820ca5 [instagram] add support for tv URLs 2019-09-21 21:57:45 +01:00
6cf6b357f5 [mixcloud] allow uppercase letters in format urls(closes #19280) 2019-09-20 11:14:24 +01:00
f455a934e9 [brightcove] delegate all supported BrightcoveLegacyIE URLs to BrightcoveNewIE
closes #11523
closes #12842
closes #13912
closes #15669
closes #16303
2019-09-19 18:02:26 +01:00
d9d3098675 [hotstar] Use native HLS downloader by default 2019-09-19 03:03:07 +07:00
1cb812d3c2 [hotstar] Extract more formats (closes #22323) 2019-09-19 03:00:19 +07:00
6fd26a7d4a [9now] Fix extraction (closes #22361) 2019-09-19 02:31:39 +07:00
9cf26b6e1d [zdf] Bypass geo restriction 2019-09-19 01:11:52 +07:00
20e11b70ac [tv4] Fix extraction and extract series metadata (closes #22443) 2019-09-18 23:45:26 +07:00
e1f692f0b3 release 2019.09.12.1 2019-09-12 02:53:52 +07:00
2f851a7d7d [ChangeLog] Actualize
[ci skip]
2019-09-12 02:48:07 +07:00
4878759f3b [youtube] Remove quality and tbr for itag 43 (closes #22372) 2019-09-12 02:46:12 +07:00
303d3e142c [ChangeLog] Actualize
[ci skip]
2019-09-12 02:05:54 +07:00
bd10b229c0 release 2019.09.12 2019-09-12 01:21:21 +07:00
035c7a59e8 [ChangeLog] Actualize
[ci skip]
2019-09-12 01:18:25 +07:00
bf1317d257 [youtube] Quick extraction tempfix (closes #22367, closes #22163) 2019-09-11 22:44:47 +07:00
bff90fc518 [youtube] Add support for invidious tor instances (#22268) 2019-09-03 01:35:32 +07:00
31dbd054c8 [platzi] Improve client data extraction (closes #22290) 2019-09-03 01:24:20 +07:00
66d04c74e0 [platzi:course] Add support for authentication 2019-09-03 01:23:22 +07:00
d7da1e37c7 [nickjr] Add support for nickelodeonjunior.fr (#22246) 2019-09-02 00:59:57 +07:00
f620d0d860 release 2019.09.01 2019-09-01 03:33:02 +07:00
79dd8884bb [ChangeLog] Actualize
[ci skip]
2019-09-01 03:18:35 +07:00
df228355fd [xhamster:user] Add extractor (closes #16330, closes #18454) 2019-09-01 03:12:56 +07:00
8945b10f6e [xhamster] Add support for more domains 2019-09-01 03:09:04 +07:00
7cb51b5daf [extractor/generic] Improve squarespace detection and fix test (closes #21859, refs #21294, refs #21802) 2019-09-01 01:25:48 +07:00
d78657fd18 [extractor/generic] Add support for squarespace embeds (closes #21294) 2019-09-01 01:25:48 +07:00
cc73d5ad15 [openload] Fix domains regex 2019-09-01 01:25:48 +07:00
71f47617c8 [downloader/external] Respect mtime option for aria2c (#22242) 2019-09-01 00:24:43 +07:00
3f46a25a97 [verystream] add support for woof.tube (closes #22217) 2019-08-31 10:02:09 +01:00
9d058b3206 [dailymotion] Add support for lequipe.fr (closes #21328, closes #22152) 2019-08-29 23:08:19 +07:00
b500955a58 [openload] Add support for oload.vip (closes #22205) 2019-08-28 01:58:07 +07:00
Jay
acc86c9a97 [bbc] Fix some tests 2019-08-28 01:53:40 +07:00
Jay
b72305f078 [bbccouk] Extend _VALID_URL (closes #19200) 2019-08-28 01:53:40 +07:00
494d664e67 [youtube] Add support for invidious.nixnet.xyz and yt.elukerio.org (#22223) 2019-08-28 01:39:59 +07:00
d1fcf255c5 [safari] Fix authentication (closes #22161) (#22184) 2019-08-27 10:16:04 +07:00
183a18c4e7 [usanetwork] Fix extraction (closes #22105) 2019-08-26 03:38:54 +07:00
393cc31d5e [einthusan] Add support for einthusan.ca (#22171) 2019-08-21 09:52:59 +07:00
0add33abcb [youtube] Improve unavailable message extraction (refs #22117) 2019-08-16 23:44:11 +07:00
0326bcb6c1 [piksel] add subtitle capability (#20506) 2019-08-15 22:14:47 +00:00
def849e0e6 release 2019.08.13 2019-08-13 23:18:38 +07:00
69611a1616 [ChangeLog] Actualize
[ci skip]
2019-08-13 23:10:05 +07:00
351f37c022 [youtube:playlist] Improve flat extraction (closes #21927) 2019-08-13 05:02:52 +07:00
3bce4ff7d9 [downloader/fragment] Fix ETA calculation of resumed download (#21992) 2019-08-11 06:57:43 +07:00
ffddb11264 [YoutubeDL] check annotations availabilty(closes #18582) 2019-08-09 08:19:41 +01:00
64b6a4e91e [youtube] fix annotations extraction(closes #22045) 2019-08-09 08:16:53 +01:00
b3d39be239 [discovery] extract series meta field(#21808) 2019-08-08 23:23:58 +01:00
1357734978 [youtube] Improve error detection (#16445) 2019-08-06 02:32:44 +07:00
eb9c9c74a6 [vimeo] fix album extraction
closes #1933
closes #15704
closes #15855
closes #18967
closes #21986
2019-08-03 10:29:20 +01:00
5efbc1366f [roosterteeth] add support for watch URLs 2019-08-02 19:38:35 +01:00
995f319b06 [discovery] limit video data by show slug(closes #21980) 2019-08-02 18:08:26 +01:00
d9d3a5a816 [README.md] Move code from #21939 to the right place 2019-08-02 05:54:56 +07:00
4f2d735803 release 2019.08.02 2019-08-02 05:37:54 +07:00
2e9522b061 [ChangeLog] Actualize
[ci skip]
2019-08-02 05:36:32 +07:00
be306d6a31 [tvigle] Fix extraction and add support for HLS and DASH formats (closes #21967) 2019-08-02 05:25:01 +07:00
33b529fabd [yandexvideo] Add support for DASH formats (#21971) 2019-08-02 05:03:25 +07:00
07f3a05c87 [CONTRIBUTING.md] Add some more coding conventions (#21939) 2019-08-02 04:49:01 +07:00
535111657b [discovery] use API call for video data extraction(#21808) 2019-08-01 22:45:10 +01:00
826dcff99c Output batch filename when it could not be read (#21915) 2019-08-01 03:54:39 +07:00
9a37ff82f1 [mgtv] Extract format_note (#21881)
format_note should now show 标清, 高清, 超清, 蓝光, etc.
2019-08-01 03:45:02 +07:00
766c4f6090 [tvn24] Fix test 2019-07-31 02:32:45 +07:00
7279163412 [tvn24] Fix metadata extraction (closes #21833, closes #21834) 2019-07-31 02:32:45 +07:00
07ab44c420 [dlive] Relax _VALID_URL (#21909) 2019-07-31 01:43:49 +07:00
2c8b1a21e8 [openload] Add support for oload.best (#21913) 2019-07-31 01:40:50 +07:00
c2d125d99f [youtube] Improve metadata extraction for age gate content (closes #21943) 2019-07-31 00:14:33 +07:00
85c2c4b4ab release 2019.07.30 2019-07-30 09:43:47 +07:00
8614a03f9c [ChangeLog] Actualize
[ci skip]
2019-07-30 09:41:23 +07:00
8dbf751aa2 [youtube] improve title and description extraction(closes #21934) 2019-07-30 00:13:33 +01:00
90634acfcf release 2019.07.27 2019-07-27 03:44:55 +07:00
eaba9dd6c2 [ChangeLog] Actualize
[ci skip]
2019-07-27 03:43:33 +07:00
843ad1796b Fix typos (#21901) 2019-07-26 22:30:18 +07:00
608b8a4300 [yahoo:japannews] Add extractor (closes #21698) (#21265) 2019-07-22 00:59:36 +07:00
ab794a553c [ctsnews] PEP 8 2019-07-21 14:59:53 +07:00
3b446ab351 [discovery] add support go.discovery.com URLs 2019-07-20 20:20:53 +01:00
13a75688a5 [youtube] Fix some tests 2019-07-21 00:01:46 +07:00
2e18adec98 [youtube:playlist] Relax _VIDEO_RE (closes #21844) 2019-07-20 23:46:34 +07:00
9c1da4a9f9 [extractor/generic] Restrict --default-search schemeless URLs detection pattern (closes #21842) 2019-07-20 23:08:26 +07:00
5e1c39ac85 [extractor/common] Fix typo in thumbnails resolution description (#21817) 2019-07-17 22:47:53 +07:00
1824bfdcdf [vrv] fix CMS signing query extraction(closes #21809) 2019-07-16 22:51:10 +01:00
2f1991ff14 release 2019.07.16 2019-07-16 00:01:46 +07:00
8b4a0ebf10 [ChangeLog] Actualize
[ci skip]
2019-07-15 23:59:23 +07:00
f61496863d [asiancrush] Add support for yuyutv.com, midnightpulp.com and cocoro.tv (closes #21281, closes #21290) 2019-07-15 23:58:08 +07:00
799756a3b3 [kaltura] Check source format URL (#21290) 2019-07-15 23:58:08 +07:00
7d4dd3e5b4 [ctsnews] Fix YouTube embeds extraction (#21678) 2019-07-15 23:03:03 +07:00
f2a213d025 [einthusan] Add support for einthusan.com (closes #21748) (#21775) 2019-07-15 22:58:55 +07:00
791d2e8117 [youtube] Add support for invidious.mastodon.host (#21777) 2019-07-15 22:54:22 +07:00
2adedc477e [gfycat] Extend _VALID_URL (closes #21779) (#21780) 2019-07-15 22:53:20 +07:00
898238e9f8 [youtube] Restrict is_live extraction (closes #21782) 2019-07-14 20:30:05 +07:00
ce80cacefd release 2019.07.14 2019-07-14 03:10:49 +07:00
0250161c52 [yandexmusic] Add missing import 2019-07-14 03:09:16 +07:00
364a2cb658 [ChangeLog] Actualize
[ci skip]
2019-07-14 03:07:02 +07:00
2fe074a960 [porn91] Fix extraction (#21312) 2019-07-14 02:57:43 +07:00
c452790a79 [yandexmusic] Add support for multi disk albums and extract track number and disk number (closes #21420) (#21421)
* [yandexmusic] extract tracks from all volumes of an album (closes #21420)

* [yandexmusic] extract genre, disk_number and track_number

* [yandexmusic] extract decomposed artist names

* Update yandexmusic.py

* Update yandexmusic.py

* Update yandexmusic.py
2019-07-14 02:38:47 +07:00
d89a0a8026 [lynda] Handle missing subtitles (closes #20490, closes #20513) 2019-07-14 01:45:28 +07:00
ba036333bf [youtube] Add more invidious instances to _VALID_URL (#21694) 2019-07-14 01:23:22 +07:00
b7ef93f0ab [twitter] Improve uploader id extraction (closes #21705) 2019-07-14 01:19:17 +07:00
f9eeeda31c [spankbang] Fix and improve metadata extraction 2019-07-14 00:21:39 +07:00
5f562bd4bb [spankbang] Fix extraction (closes #21763, closes #21764) 2019-07-14 00:13:26 +07:00
b99f11a56b [dlive] restrict DLive Stream _VALID_URL regex 2019-07-13 14:11:57 +01:00
4a71ef6da6 [dlive] Add new extractor(closes #18080) 2019-07-13 13:08:19 +01:00
fd95105ed4 [livejournal] Add new extractor(closes #21526) 2019-07-13 12:47:02 +01:00
c72dc20d09 [roosterteeth] fix free episode extraction(#16094) 2019-07-13 10:13:07 +01:00
272355c172 [dbtv] fix extraction 2019-07-12 23:26:46 +01:00
57227618fe [spike] fix Bellator extraction 2019-07-12 22:50:37 +01:00
0441d6266c [rudo] remove extractor(closes #18430)(closes #18474)
Covered by generic extractor
2019-07-12 22:31:11 +01:00
82f68e4a01 [facebook] fallback to twitter:image meta for thumbnail extraction(closes #21224) 2019-07-12 22:02:06 +01:00
d4ece5d359 [bleacherreport] fix Bleacher Report CMS extraction 2019-07-12 21:56:49 +01:00
16d3672ad7 [espn] fix fivethirtyeight.com extraction 2019-07-11 23:37:34 +01:00
0dd58a523f [fivetv] relax video URL regex and support https URLs 2019-07-11 23:10:35 +01:00
27019dbb4b [youtube] Fix is_live extraction (closes #21734) 2019-07-12 03:45:58 +07:00
baf67a604d [youtube] Fix authentication (closes #11270) 2019-07-12 02:26:05 +07:00
0d1f4af39d release 2019.07.12 2019-07-12 00:43:54 +07:00
7612406bf9 [ChangeLog] Actualize
[ci skip]
2019-07-12 00:34:03 +07:00
4dcd4b7b16 [mgtv] Pass Referer HTTP header for format URLs (closes #21726) 2019-07-12 00:04:25 +07:00
5fc0896168 [beeg] Add support for api/v6 v2 URLs without t argument (closes #21701) 2019-07-11 23:37:09 +07:00
e4d53148f5 [funnyordie] move extraction to VoxMedia extractor and improve vox volume embed extraction(closes #16846) 2019-07-10 16:47:37 +01:00
cfe781d4fa [gameinformer] fix extraction(closes #8895)(closes #15363)(closes #17206) 2019-07-10 15:45:00 +01:00
253289656f [extractors] update funk.net import 2019-07-10 13:57:43 +01:00
4b30282616 [funk] fix extraction(closes #17915) 2019-07-10 13:54:49 +01:00
c9b0564ac1 [packtpub] Relax lesson _VALID_URL regex(closes #21695) 2019-07-09 11:56:16 +01:00
25d71fb058 [packtpub] fix extraction(closes #21268) 2019-07-09 08:28:56 +01:00
a6389abfd7 [philharmoniedeparis] Relax _VALID_URL (closes #21672) 2019-07-06 23:17:45 +07:00
d18003a141 [peertube] Detect embed URLs in generic extraction (closes #21666) 2019-07-06 00:50:56 +07:00
d1850c1a97 [mixer:vod] Relax _VALID_URL (closes #21657) (#21658) 2019-07-05 22:47:32 +07:00
c9fa84d88e [lecturio] add support id based URLs(closes #21630) 2019-07-04 15:59:45 +01:00
a30c2f4055 [go] Add site info for disneynow (closes #21613) 2019-07-04 04:01:30 +07:00
5ae9b8b3a3 [adobepass] Add support for AT&T U-verse (mso ATT) (closes #13938, closes #21016) 2019-07-04 03:57:11 +07:00
cdb7c7d147 [ted] Restrict info regex (closes #21631) 2019-07-04 02:04:23 +07:00
2da4316e48 [twitch:vod] Actualize m3u8 URL (#21538, #21607) 2019-07-03 23:22:23 +07:00
313877c6a2 [vzaar] Fix videos with empty title (closes #21606) 2019-07-03 23:16:40 +07:00
e61ac1a09c [tvland] fix extraction(closes #21384) 2019-07-03 13:31:47 +01:00
ff0f4cfeba [arte] clean extractor(closes #15583)(closes #21614) 2019-07-02 22:09:40 +01:00
1335bf10f6 release 2019.07.02 2019-07-02 01:09:59 +07:00
c8343f0a43 [ChangeLog] Actualize
[ci skip]
2019-07-02 01:07:54 +07:00
d1e4116427 [vevo] Add support for embed.vevo.com URLs (#21565) 2019-07-02 00:13:23 +07:00
9baf69af45 [openload] Add support for oload.biz (#21574) 2019-07-02 00:11:38 +07:00
Fai
918398092c [xiami] Update API base URL (#21575) 2019-07-02 00:10:55 +07:00
4e2491f066 [yourporn] Fix extraction (#21585) 2019-07-02 00:05:51 +07:00
976e1ff7f9 [acast] add support for URLs with episode id(closes #21444) 2019-07-01 12:05:18 +01:00
5e3da0d42b [dailymotion] add support embed with DM.player js call 2019-07-01 08:37:21 +01:00
c560680247 [soundcloud] Update client id 2019-06-29 00:33:35 +07:00
f7a147e3b6 [utils] Introduce random_user_agent and use as default User-Agent (closes #21546) 2019-06-29 00:32:43 +07:00
8c8cae91ec release 2019.06.27 2019-06-27 23:57:33 +07:00
232331c0d2 [ChangeLog] Actualize
[ci skip]
2019-06-27 23:55:15 +07:00
4f71473ef1 [go] Add support for disneynow.com (closes #21528) 2019-06-27 22:59:30 +07:00
6625bf200d [mixer:vod] Relax _VALID_URL (closes #21531) (#21536) 2019-06-27 22:24:46 +07:00
f562994660 [drtv] Relax _VALID_URL 2019-06-27 22:18:10 +07:00
509bcec37b [fusion] fix extraction(closes #17775)(closes #21269) 2019-06-27 12:06:31 +01:00
1d83e9bd4b [nfb] Remove extractor (closes #21518)
Covered by generic extractor
2019-06-25 00:12:31 +07:00
27cef8885d [beeg] Add support for api/v6 v2 URLs (closes #21511) 2019-06-24 23:01:52 +07:00
3031b7c4ed [brightcove:new] Add support for playlists (#21331) 2019-06-23 17:04:05 +07:00
695720ebe8 [openload] Add support for oload.life (#21495) 2019-06-23 04:31:43 +07:00
2605043d6d [vimeo:channel,group] Make title extraction no fatal 2019-06-23 02:16:09 +07:00
091c9b4316 [vimeo:likes] Implement extrator in terms of channel extractor
This allows to obtain videos' ids before extraction (#21493)
2019-06-23 02:13:46 +07:00
9634de178d [pornhub] Add support for more paged video sources 2019-06-22 08:37:07 +07:00
1f7a563ab0 [pornhub] Add support for downloading single pages and search pages (closes #15570) 2019-06-22 06:01:43 +07:00
21b08463a7 [pornhub] Rework extractors (closes #11922, closes #16078, closes #17454, closes #17936) 2019-06-22 05:34:46 +07:00
31ce6e9966 [youtube] Add another signature function pattern 2019-06-22 02:22:41 +07:00
1c11204056 [tf1] Improve extraction and fix issues (closes #21372) 2019-06-22 00:40:06 +07:00
9c2aaac268 [tf1] Fix wat id extraction (closes #21365) 2019-06-22 00:40:00 +07:00
d415957dbc [crunchyroll] Move Accept-Language workaround to video extractor since it causes playlists not to list any videos 2019-06-22 00:15:52 +07:00
4681441d2f [crunchyroll:playlist] Fix and relax title extraction (closes #21291, closes #21443) 2019-06-22 00:07:26 +07:00
9842d29d66 release 2019.06.21 2019-06-21 23:04:09 +07:00
bc6438c092 [ChangeLog] Actualize
[ci skip]
2019-06-21 23:01:09 +07:00
abefc03f51 [youtube] Update signature function patterns (closes #21469, closes #21476) 2019-06-21 22:58:42 +07:00
c40714cdee [youtube] Make --write-annotations non fatal (closes #21452) 2019-06-20 00:57:58 +07:00
7c24a58bdb [sixplay] add support for rtlmost.hu(#21405) 2019-06-16 06:32:47 +01:00
b85eae0f05 [youtube] Hardcode codec metadata for av01 video only formats (closes #21381) 2019-06-14 01:59:05 +07:00
28cc2241e4 [utils] Restrict parse_codecs and add theora as known vcodec (#21381) 2019-06-14 01:56:17 +07:00
8361e7f934 [toutv] update client key(closes #21370) 2019-06-12 21:41:46 +01:00
427cc21531 [biqle] remove unnecessary regex group 2019-06-10 15:17:26 +01:00
f991dd2bfb [biqle] add support for new embed domain 2019-06-10 14:58:37 +01:00
35c2dd48d9 [cbs] improve DRM protected videos detection(closes #21339) 2019-06-09 00:13:07 +01:00
3b2fd09596 release 2019.06.08 2019-06-08 03:47:30 +07:00
e35fc5ebc7 [ChangeLog] Actualize
[ci skip]
2019-06-08 03:45:31 +07:00
dbb1886114 [ted] Improve playlist extraction (closes #21032) 2019-06-08 03:07:08 +07:00
c2ee6fa66a [ted] Fix playlist extraction (closes #20844) 2019-06-08 03:07:02 +07:00
4831ef7fe4 [vlive] Add comments to tests and quotes consistency 2019-06-08 02:42:01 +07:00
178663df52 [vlive:playlist] Redirect to video extractor when no playlist is found (#20590) 2019-06-08 02:40:47 +07:00
ef19739e64 [vlive] Improve and style (closes #21209) 2019-06-08 02:22:23 +07:00
01b517a20a [vlive] Add CH+ support (closes #16887) 2019-06-08 02:22:15 +07:00
f4c99cd635 [openload] Add support for oload.website (#21329) 2019-06-08 02:03:58 +07:00
e75220b11a [tvnow] Extract HD formats (closes #21201) 2019-06-08 00:45:28 +07:00
2efefddafd [redbulltv:rrn:content] Add extractor (closes #21297) 2019-06-07 23:48:57 +07:00
bf3c932663 [youtube] Fix average rating extraction (closes #21304) 2019-06-07 23:13:33 +07:00
4c78c3d700 [bitchute] Extract HTML5 formats (closes #21306) 2019-06-07 22:58:19 +07:00
2e11e51c04 [cbsnews] fix extraction(closes #9659)(closes #15397) 2019-06-05 01:23:23 +01:00
1a01639bf9 [downloader/common] Improve rate limit (#21301) 2019-06-05 03:06:35 +07:00
59ca17b1c8 [vvvvid] relax _VALID_URL(closes #21299) 2019-06-04 21:03:52 +01:00
c94c121a99 Revert "[southpark] add support for videos with english audio track"
This reverts commit 0c84002650.
2019-06-04 00:37:58 +01:00
0c84002650 [southpark] add support for videos with english audio track(closes #21271) 2019-06-03 12:43:39 +01:00
c5eb75b35a [prosiebensat1] add support for new API(closes #21272) 2019-06-01 15:12:30 +01:00
0e2dd3fcbc [vrv] extract adaptive_hls formats(closes #21243) 2019-06-01 11:17:44 +01:00
26a87972a9 [viki] Switch to HTTPS (closes #21001) 2019-05-29 04:42:19 +07:00
33b2218b2f [LiveLeak] Check if the original videos exist (closes #21206) (#21208) 2019-05-28 12:31:11 +00:00
ead467a9c1 [rtp] fix extraction(closes #15099) 2019-05-28 04:58:12 +01:00
0d29751890 [youtube] improve DRM protected videos detection(#1774) 2019-05-25 23:15:59 +01:00
25b83c2a0e [srgssrplay] Improve _VALID_URL (closes #21155) 2019-05-24 00:45:06 +07:00
3fe774722b [srgssrplay] Add support for popupvideoplayer URLs 2019-05-24 00:44:37 +07:00
f4cc2ca503 [24video] Add support for porno.24video.net (closes #21194) 2019-05-24 00:38:06 +07:00
11ec06de7f [24video] Add support for 24video.site (closes #21193) 2019-05-24 00:35:46 +07:00
53cd37bac5 [utils] Improve strip_or_none 2019-05-24 00:03:01 +07:00
f856816b94 [extractor/common] Strip src attribute for HTML5 entries code (closes #18485, closes #21169) 2019-05-23 23:52:11 +07:00
8af49fc276 [pornflip] Remove extractor 2019-05-23 23:48:06 +07:00
9c5f2988b9 [criterion] Remove extractor (closes #21195) 2019-05-23 23:38:01 +07:00
afd4985f72 [travis] Force dist to Ubuntu Trusty by default
According to https://blog.travis-ci.com/2019-04-15-xenial-default-build-environment Ubuntu Xenial is now default, but it lacks python 2.6, 3.2 and 3.3 support needed by tests
2019-05-23 06:06:49 +07:00
2c53c0ebc6 [pornhub] Use https (#21061) 2019-05-23 03:56:54 +07:00
bbf1defe58 [bitchute] Fix uploader extraction (#21076) 2019-05-23 03:51:50 +07:00
186d185b6e [streamcloud] Reduce waiting time to 6 seconds (#21092) 2019-05-23 03:46:20 +07:00
612300a686 [novamov] Remove extractors (#21077)
Sites no longer exist
2019-05-23 03:38:48 +07:00
ea75382094 [openload] Add support for oload.press (#21135) 2019-05-23 03:30:17 +07:00
e438e81469 [vivo] Improve extraction (closes #19217) 2019-05-23 03:06:07 +07:00
0e6f914b3b [vivo] Fix extraction (closes #18906) 2019-05-23 03:06:07 +07:00
6ab30ff50b release 2019.05.20 2019-05-20 23:29:49 +07:00
42c971341b [ChangeLog] Actualize
[ci skip]
2019-05-20 23:24:27 +07:00
ce2fe4c01c [extractor/common] Add doc string for _apply_first_set_cookie_header 2019-05-20 23:23:18 +07:00
a9e03736df [safari] Fix authentication (closes #21090) 2019-05-18 03:33:40 +07:00
e3c1266f49 [extractor/common] Move workaround for applying first Set-Cookie header into a separate method 2019-05-18 03:17:15 +07:00
82e91d20a0 [vrt] fix extraction(closes #20527) 2019-05-15 15:02:51 +01:00
170d644440 [canvas] add support for vrtnieuws and sporza site ids and extract AES HLS formats 2019-05-15 14:58:57 +01:00
5831742840 [vrv] extract captions(closes #19238) 2019-05-15 10:39:59 +01:00
a277dd33eb [tele5] Improve video id extraction 2019-05-13 00:13:00 +07:00
0e0bfd334c [tele5] Relax _VALID_URL (closes #21020, closes #21063) 2019-05-13 00:11:48 +07:00
e6a25fea23 [svtplay] Update API URL (closes #21075) 2019-05-12 23:24:01 +07:00
b7df8f90a7 [yahoo:gyao] add X-User-Agent header to dam proxy requests(closes #21071) 2019-05-12 10:23:36 +01:00
8ae32e1882 release 2019.05.11 2019-05-11 06:07:28 +07:00
07a2cef782 [ChangeLog] Actualize
[ci skip]
2019-05-11 06:04:53 +07:00
8ed7a23328 [extractor/common] Fix typo 2019-05-11 04:53:48 +07:00
3089bc748c Fix W504 and disable W503 (closes #20863) 2019-05-11 03:57:40 +07:00
d23e85515a [tvnow] Fix W605 2019-05-11 03:57:40 +07:00
cd37ef44e9 [commonmistakes] Fix E117 2019-05-11 03:57:40 +07:00
169f8d0fe1 [cloudflarestream] Add support for videodelivery.net (#21049) 2019-05-11 03:09:00 +07:00
0db2b275dd [byutv] Improve extraction and update DVR test (closes #20676) 2019-05-11 03:06:18 +07:00
ab11674502 [byutv] Add support for DVR videos (closes #20574)
Fix code style on brackets (flake8)

Add more information to test info_dict
2019-05-11 03:06:12 +07:00
68b92aa1b4 [gfycat] Add support for URLs with tags (closes #20696) (#20731) 2019-05-11 02:12:45 +07:00
fd35d8cdfd [utils] Transliterate "þ" as "th" (#20897)
Despite visual similarity "þ" is unrelated to "p".
It is normally transliterated as "th":

    $ echo þ-Þ | iconv -t ASCII//TRANSLIT
    th-TH
2019-05-11 01:42:31 +07:00
4eec112740 [openload] Add support for verystream.com (closes #20701) (#20967) 2019-05-11 01:35:56 +07:00
027ffdca0d [youtube] Use sp field value for signature field name (closes #18841, closes #18927, closes #21028) 2019-05-10 08:38:05 +07:00
a5b92d3590 [yahoo:gyao] extend _VALID_URL(closes #21008) 2019-05-09 04:14:19 +01:00
b45a9e698e [youtube] Fix channel id extraction (closes #20982) (#21003) 2019-05-09 00:58:47 +07:00
71ebd35d50 [sky] add support for news.sky.com (closes #13055) 2019-05-07 10:17:18 +01:00
f8c55c6664 [youtube:entrylistbase] Retry on 5xx HTTP errors (#20965) 2019-05-06 01:25:16 +07:00
876fed6bf3 [francetvinfo] Extend video id extraction (closes #20619) (#20740) 2019-05-05 06:26:30 +07:00
c9856648db [4tube] Update token hosts (closes #20918) 2019-05-04 03:26:40 +07:00
2533f5b691 [hotstar] Move to API v2 (closes #20931) 2019-05-04 03:11:25 +07:00
e0dde1d8e2 [fox] fix Uplynk PrePlay error handling under python 2(#20925) 2019-05-02 10:46:29 +01:00
62d10f0d32 [fox] Fix API error handling under python 2 (closes #20925) 2019-05-02 00:36:52 +07:00
274519dd08 [redbulltv] Extend _VALID_URL (closes #20922) 2019-05-01 21:36:19 +07:00
091200c368 release 2019.04.30 2019-04-30 06:11:50 +07:00
67bfbe4942 [ChangeLog] Actualize
[ci skip]
2019-04-30 06:08:51 +07:00
54f3b61216 [openload] Use real Chrome versions (closes #20902) 2019-04-30 05:59:12 +07:00
a61ce71468 [youtube] Remove info el for get_video_info request
Since it does not work for quite a long time
2019-04-30 04:49:12 +07:00
026fbedc85 [youtube] Improve extraction robustness
Fail on missing token only when no formats found
2019-04-30 04:32:55 +07:00
6e07b5a6d5 [dramafever] Remove extractor(closes #20868) 2019-04-28 18:02:41 +01:00
c464e1df2c [adn] fix subtitle extraction(#12724) 2019-04-28 17:50:47 +01:00
92bc97d398 [youtube] extract album from Music in this video section(#20301) 2019-04-28 17:38:20 +01:00
f916abc0ac [ccc] Improve extraction (closes #14601, closes #20355) 2019-04-28 23:08:09 +07:00
24510bdcfa [ccc] Extract creator 2019-04-28 23:07:41 +07:00
ae8c13565e [ccc:playlist] Add extractor 2019-04-28 23:07:01 +07:00
280913800d [sverigesradio] improve extraction(closes #18635) 2019-04-28 12:03:39 +01:00
7ff8ad80f1 [sverigesradio] Add extractor 2019-04-28 10:07:06 +01:00
4e4db743e7 [cinemax] Add new extractor 2019-04-28 00:42:55 +01:00
3545d38bfb [sixplay] add missing parenthesis 2019-04-27 10:32:53 +01:00
2309d6bf92 [sixplay] try to extract non drm protected manifests(closes #20849) 2019-04-27 10:17:34 +01:00
822b9d9cb0 [youtube] improve Youtube Music Auto-generated description parsing(closes #20742) 2019-04-27 09:16:17 +01:00
5caabd3c70 [youtube] Extract additional meta data from video description on youtube music videos
YouTube music videos often have auto-generated video descriptions that can be
utilized to extract additional information about the video. This is desirable
in order to provide the user with as much meta data as possible. This commit
adds extraction methods for the following fields for youtube music videos:
- artist (fallback extraction methods added)
- track (fallback extraction methods added)
- album (new in this commit)
- release_date (new in this commit)
- release_year (new in this commit)

4 tests have been added to test this new functionality:
- YoutubeIE tests 27, 28, 29, and 30

Resolves: #20599
2019-04-27 09:09:54 +01:00
aa05a093bb [wrzuta] Remove extractor (closes #20684) (#20801)
Wrzuta.pl was shut down in 2017.
2019-04-27 05:12:15 +07:00
60e67c5b2c [twitch] Prefer source format (closes #20850) 2019-04-27 05:08:27 +07:00
eefa0f2157 Move issue template templates into separate folder 2019-04-27 04:55:30 +07:00
6f366ef30c Issue template overhaul 2019-04-27 04:50:47 +07:00
88b547492f [twitcasting] Add support for private videos (#20843) 2019-04-26 16:17:40 +00:00
00a9a25cf9 [twitcasting] Fix test: video title (#20840) 2019-04-26 09:34:23 +00:00
97abf05ad3 [reddit] check thumbnail URL(closes #20030) 2019-04-26 10:26:51 +01:00
da668a23bd [ISSUE_TEMPLATE.md] Add entry on argument escaping in make-sure checklist
[ci skip]
2019-04-26 00:46:41 +07:00
58ef5e7881 [yandexmusic] fix track url extraction(closes #20820) 2019-04-25 11:36:44 +01:00
3e7ec5330a release 2019.04.24 2019-04-24 10:05:54 +07:00
98933c14e1 [ChangeLog] Actualize
[ci skip]
2019-04-24 10:05:08 +07:00
56667d622c [youtube] Fix extraction (closes #20758, closes #20759, closes #20761, closes #20762, closes #20764, closes #20766, closes #20767, closes #20769, closes #20771, closes #20768, closes #20770) 2019-04-24 09:58:00 +07:00
50d660479d [toutv] fix extraction and extract series info(closes #20757) 2019-04-24 00:28:00 +01:00
1fa8893734 [vrv] add support for movie listings(closes #19229) 2019-04-22 23:50:37 +01:00
15be3eb5e5 [youtube] raise ExtractorError when no data available(#20737) 2019-04-22 20:52:43 +01:00
e09965d550 [soundcloud] Add support for new rendition and improve extraction (closes #20699) 2019-04-23 00:39:36 +07:00
3fd86cfe13 [ooyala] add support for geo verification proxy 2019-04-22 10:04:56 +01:00
fdc2183650 [nrl] Add new extractor(closes #15991) 2019-04-22 10:04:00 +01:00
85b6335d55 [vimeo] extract live archive source format(#19144) 2019-04-21 21:05:58 +01:00
c25720ef6a [vimeo] add support live streams and improve info extraction(closes #19144) 2019-04-21 17:20:52 +01:00
c9b19d7a55 [ntvcojp] Add new extractor 2019-04-21 14:51:26 +01:00
47cfa00516 [nhk] extract rtmpt format 2019-04-21 13:25:04 +01:00
061d1cd948 [nhk] add support for audio URLs 2019-04-21 13:17:22 +01:00
5de538787d [udemy] add another course id extraction pattern(closes #20491) 2019-04-19 20:44:59 +01:00
9abeefd527 [openload] Add support for oload.services (closes #20691) 2019-04-18 23:56:20 +07:00
f3914b06a0 [openload] Add support for openloed.co (closes #20691)
While the .co could be captured directly, I anticipate that there will be more TLD's for openloed in the future.
2019-04-18 01:51:32 +07:00
81d989c21e [bravotv] fix extraction(closes #19213) 2019-04-18 01:50:30 +07:00
cd6c75b05f release 2019.04.17 2019-04-18 01:50:25 +07:00
9846935256 [ChangeLog] Actualize
[ci skip]
2019-04-17 00:15:48 +07:00
7fc3b68ad3 [openload] Randomize User-Agent (closes #20688) 2019-04-17 00:08:50 +07:00
c4341ea47e [openload] Add support for oladblock domains (#20471) 2019-04-16 23:50:04 +07:00
e6c9ae31df [adn] fix subtitle extraction(#12724) 2019-04-16 13:04:13 +01:00
6104cc1591 [openload] add test for oladblock.me 2019-04-15 23:09:05 -04:00
f114e43d38 [openload] add oladblock.me domain 2019-04-15 23:08:28 -04:00
cb6cd76f7b [aol] add support for localized websites 2019-04-14 23:18:36 +01:00
0b758fea1c [yahoo] add support GYAO episode URLs 2019-04-14 15:01:01 +01:00
3534b6329a [yahoo] add support for streaming.yahoo.co.jp(closes #5811)(closes #7098) 2019-04-14 14:39:20 +01:00
174f62992d [yahoo] add support for gyao.yahoo.co.jp 2019-04-14 14:29:04 +01:00
1038532213 [aenetworks] add encoding declaration 2019-04-14 13:18:16 +01:00
4f1e02ad60 [aenetworks] fix history topic extraction and extract more formats 2019-04-14 11:46:33 +01:00
180a9dff1f [cbs] extract smpte and vtt subtitles 2019-04-13 17:02:22 +01:00
972d2dd0bc [streamango] add support for streamcherry.com (#20592) 2019-04-13 15:05:24 +07:00
11edb76610 [README.md] Rephrase usage example comment (#20614) 2019-04-13 15:03:42 +07:00
8721b09751 [yourporn] Add support for sxyprn.com (#20646) 2019-04-13 15:02:09 +07:00
dc27fd8bb8 [mgtv] fix extraction(closes #20650) 2019-04-12 09:19:09 +01:00
c912029480 [linkedin:learning] use urljoin for form action url(closes #20431) 2019-04-11 08:44:58 +01:00
118f7add3b [gdc] add support for kaltura embeds and update tests(closes #20575) 2019-04-09 11:23:47 +01:00
4bc12b8f81 [dispeak] improve mp4 bitrate extraction 2019-04-09 11:21:46 +01:00
5ca3459828 [kaltura] sanitize embed URLs 2019-04-09 11:20:26 +01:00
9c017253e8 [jwplatfom] do not match manifest URLs(#20596) 2019-04-08 16:34:03 +01:00
9045d28b5e [aol] restrict url regex and improve format extraction 2019-04-07 21:31:26 +01:00
7c2ecbc1cc [tiktok] Add support for new URL schema (closes #20573) 2019-04-07 21:06:09 +07:00
d562cac9dc [stv:player] Add new extractor(closes #20586) 2019-04-07 12:40:14 +01:00
9ed06812ec [streamango] add support for streamcherry.com 2019-04-06 23:59:41 -04:00
bf6fb8b9dc [openload] add tests 2019-04-06 23:38:40 -04:00
a46d9e5b41 release 2019.04.07 2019-04-07 04:19:46 +07:00
aa5338118e [ChangeLog] Actualize
[ci skip]
2019-04-07 04:16:45 +07:00
8410653f24 [ruutu] Add support for audio podcasts (closes #20473, closes #20545) 2019-04-07 03:18:10 +07:00
f4da808036 [xvideos] Extract all thumbnails (closes #20432) 2019-04-07 02:59:09 +07:00
f412970164 [README.md] Fix lists formatting (closes #20558)
Lists have to be separated from the previous paragraph by a blank line
in certain variants of Markdown, otherwise they are not interpreted as
lists.

This change ensures that that the youtube-dl.1 man page, which is
generated from README.md with the help of pandoc, is formatted
correctly.
2019-04-07 02:28:31 +07:00
059cd768b9 [vk] Remove unused import 2019-04-07 02:17:54 +07:00
c701472fc9 [platzi] Add extractor (closes #20562) 2019-04-07 02:15:52 +07:00
19591facea [dvtv] remove unnecessary comments and spaces 2019-04-06 16:45:11 +01:00
b9aad6c427 [dvtv] Fix extraction (closes #18514) 2019-04-06 16:37:07 +01:00
9f182c23ba [vrv] add basic support for individual movie links(#19229) 2019-04-06 09:22:25 +01:00
4810655cd6 [bfi:player] Add new extractor(#19235) 2019-04-05 19:35:35 +01:00
a7978f8e2a [hbo] fix extraction and extract subtitles(closes #14629)(closes #13709) 2019-04-05 18:08:43 +01:00
19041a3877 [youtube] extract srv[1-3] subtitle formats(#20566) 2019-04-05 16:18:57 +01:00
afb7496416 [adultswim] fix extraction(closes #18025) 2019-04-05 11:45:49 +01:00
69e6efac16 [teamcoco] fix extraction and add suport for subdomains(closes #17099)(closes #20339) 2019-04-05 08:26:04 +01:00
2bbde1d09a [adn] fix subtitle compatibility with ffmpeg 2019-04-04 17:59:20 +01:00
b966740cf7 [adn] fix extraction and add support for positioning styles(closes #20549) 2019-04-04 14:50:16 +01:00
220828f2d6 [vk] use a more unique video id(closes #17848) 2019-04-03 11:08:42 +01:00
977a782110 [rtl2] update player_url 2019-04-03 10:20:01 +01:00
a2b6f946f1 [newstube] fix extraction 2019-04-03 10:19:36 +01:00
4f7db46887 [rtl2] improve _VALID_URL regex 2019-04-03 01:00:02 +01:00
d7d86fdd49 [download/external] pass rtmp_conn to ffmpeg 2019-04-02 22:41:23 +01:00
f8987163fb [adobeconnect] Add new extractor(closes #20283) 2019-04-02 22:40:39 +01:00
313e8b2b18 [gaia] add support for authentication(closes #14605) 2019-04-02 15:50:06 +01:00
c0b7d11713 [YoutubeDL] Add ffmpeg_location to post processor options (closes #20532) 2019-04-02 01:29:44 +07:00
efee62ac7f [mediasite] Add support for dashed ids and named catalogs (closes #20531) 2019-04-02 01:13:52 +07:00
38287d251d release 2019.04.01 2019-04-01 23:55:17 +07:00
25d9243141 [ChangeLog] Actualize
[ci skip]
2019-04-01 23:53:28 +07:00
93bb6b1bae [weibo] Extend _VALID_URL (#20496) 2019-03-31 01:31:33 +07:00
b43c5f474a [xhamster] Add support for xhamster.one (closes #20508) 2019-03-31 01:27:45 +07:00
4014a48622 [mediasite:catalog] Add extractor (closes #20507) 2019-03-31 01:21:53 +07:00
99fe330070 [teamtreehouse] Add new extractor(closes #9836) 2019-03-28 16:55:57 +01:00
c4c888697e [ina] add support for audio URLs 2019-03-27 18:49:29 +01:00
b27a71e66c [ina] improve extraction 2019-03-27 18:29:24 +01:00
de74ef83b7 [cwtv] fix episode number extraction(closes #20461) 2019-03-27 18:01:51 +01:00
cf3d399727 [openload] add support for oladblock.services and oladblock.xyz domains 2019-03-25 12:04:31 -04:00
8cb10807ed [npo] Improve DRM detection 2019-03-23 21:43:50 +07:00
b8526c78f9 [pornhub] Add support for DASH formats (closes #20403) 2019-03-23 01:09:33 +07:00
5e1271c56d [utils] Improve int_or_none and float_or_none (#20403) 2019-03-23 01:08:54 +07:00
050afa60c6 Check for valid --min-sleep-interval when --max-sleep-interval is specified 2019-03-21 22:55:03 +07:00
c4580580f5 [svtplay] Update API endpoint (closes #20430) 2019-03-21 22:39:35 +07:00
0a8e251b35 release 2019.03.18 2019-03-18 01:36:41 +07:00
2ed2ebdb36 [ChangeLog] Actualize
[ci skip]
2019-03-18 01:33:37 +07:00
8428fdccf2 [yandexvideo] Add extractor 2019-03-17 17:33:07 +07:00
04988b55b5 [openload] Improve embed detection 2019-03-17 15:34:21 +07:00
034f5fb5ee [radiocanada:audiovideo] Fix typo 2019-03-17 14:34:48 +07:00
a63782b581 [corus] Add support for bigbrothercanada.ca (#20357) 2019-03-17 14:20:21 +07:00
0146c6cde6 [orf:radio] Extract series (#20012) 2019-03-17 09:57:02 +07:00
ddff25c5d1 [extractors] Remove superfluous whitespace 2019-03-17 09:41:16 +07:00
81dada0b4b [cbc:watch] Add support for gem.cbc.ca (closes #20251, #20359) 2019-03-17 09:37:29 +07:00
eba3a2f9ef [anysex] Remove extractor (closes #19279) 2019-03-17 09:15:24 +07:00
d493f15c11 [extractor/common] Improve HTML5 entries extraction and add some realworld tests 2019-03-17 09:09:32 +07:00
0dc41787af [utils] Introduce parse_bitrate 2019-03-17 09:07:47 +07:00
e5cfb779ea [ciscolive] Add support for new URL schema (closes #20320, #20351) 2019-03-17 07:18:54 +07:00
77d95677b7 [youtube] Add support for invidiou.sh (#20309) 2019-03-17 07:15:15 +07:00
ffbd1368df [update] Hide update URLs behind redirect 2019-03-17 07:08:20 +07:00
2e27421c70 [test_InfoExtractor] Add test for #20346 2019-03-15 01:20:24 +07:00
79d2077edc [extractor/common] Fix url meta field for unfragmented DASH formats (closes #20346) 2019-03-15 00:42:14 +07:00
6db03a29d1 [anitube] Remove extractor
site no longer exists
2019-03-13 10:47:55 +01:00
47f9792620 [ruleporn] Remove extractor (closes #15344, closes #20324)
Covered by generic extractor
2019-03-12 22:55:32 +07:00
68867668cf [npr] fix extraction(closes #10793)(closes #13440) 2019-03-12 16:52:28 +01:00
e7e3ec828b [biqle] fix extraction(closes #11471)(closes #15313) 2019-03-12 08:08:54 +01:00
e585547284 [viddler] Modernize 2019-03-11 15:53:44 +01:00
4993132344 [pandatv] fix condition syntax 2019-03-11 15:37:19 +01:00
8569058f88 [moevideo] fix extraction 2019-03-11 15:18:09 +01:00
612a159510 [primesharetv] remove extractor 2019-03-11 09:51:26 +01:00
f517c0c0a7 [hypem] Modernize and extract more metadata(closes #15320) 2019-03-11 09:43:06 +01:00
197cf8b794 [veoh] fix extraction 2019-03-11 09:04:06 +01:00
fddb4a3a8b [escapist] Modernize 2019-03-11 08:18:57 +01:00
99534feaff [videomega] remove extractor(closes #10108) 2019-03-11 07:27:57 +01:00
cdc7baffca [beeg] Add support for beeg.porn (closes #20306) 2019-03-11 04:44:40 +07:00
067aa17edf Start moving to ytdl-org 2019-03-11 04:00:54 +07:00
2765503713 [vimeo:review] improve config url extraction and extract original format(closes #20305) 2019-03-10 15:03:32 +01:00
0d08bcdb70 [fox] detect geo restriction and authentication errors(#20208) 2019-03-10 09:37:40 +01:00
10734553fe release 2019.03.09 2019-03-09 02:53:18 +07:00
bba35695eb [ChangeLog] Actualize
[ci skip]
2019-03-09 02:52:08 +07:00
9d74ea6d36 [francetv:site] Relax video id regex and update test (closes #20268) 2019-03-08 23:28:24 +07:00
7b6e760870 [toutv] detect invalid login error 2019-03-06 09:28:14 +01:00
829685b88a [toutv] fix authentication(closes #20261) 2019-03-06 09:20:39 +01:00
fca9baf0da [test] Fix test_compat_etree_Element 2019-03-06 02:46:26 +07:00
d347b52b63 [urplay] Extract timestamp (#20235) 2019-03-06 02:11:32 +07:00
97157c692c [openload] Add support for oload.space 2019-03-06 01:34:34 +07:00
a551768acf [facebook] Improve uploader extraction (closes #20250) 2019-03-06 01:27:22 +07:00
ee0ba927aa Use compat_etree_Element 2019-03-06 01:21:57 +07:00
399f76870d [compat] Introduce compat_etree_Element 2019-03-06 01:18:52 +07:00
e5ada4f3ad [extractor/common] Fallback url to base URL for DASH formats 2019-03-06 00:33:08 +07:00
bb6f112d9d [npo] Improve ISM extraction 2019-03-05 23:58:46 +07:00
c17eb5b4b0 [rai] Improve extraction (closes #20253) 2019-03-05 23:56:42 +07:00
d9eb580a79 [extractor/common] Do not fail on invalid data while parsing F4M manifest in non fatal mode 2019-03-05 23:56:33 +07:00
5dcd630dca [paramountnetwork] fix mgid extraction(closes #20241) 2019-03-04 22:26:55 +01:00
c790e93ab5 [extractor/common] Clarify url and manifest_url meta fields 2019-03-05 00:41:53 +07:00
39c780fdec [extractor/common] Return MPD manifest as format's url meta field (#20242)
For symmetry with other segmented media
2019-03-05 00:40:57 +07:00
e7e62441cd [utils] strip #HttpOnly_ prefix from cookies files (#20219) 2019-03-03 19:23:59 +07:00
0a5baf9c21 [libsyn] improve extraction(closes #20229) 2019-03-03 06:18:51 +01:00
8ae113ca9d [youtube] Add more invidious instances
See [Invidious-Instances](https://github.com/omarroth/invidious/wiki/Invidious-Instances) for the reference.
2019-03-03 08:19:36 +07:00
7aeb788e56 [travis] Remove sudo: false
Travis now recommends removing `sudo: false` from configuration: https://blog.travis-ci.com/2018-11-19-required-linux-infrastructure-migration.
2019-03-03 08:16:48 +07:00
7465e0aee2 [spankbang] Fix extraction (closes #20023) 2019-03-03 06:25:45 +07:00
a8f83f0c56 [crunchyroll] fix is_logged check 2019-03-02 08:25:47 +01:00
dca0e0040a Revert "use older login method(closes #11572)"
This reverts commit cc6a960e13.
2019-03-02 08:01:42 +01:00
398e1e21d6 [espn] extend _VALID_URL regex(closes #20013) 2019-03-01 15:34:05 +01:00
c5b02efe20 [sixplay] handle videos with empty assets(closes #20016) 2019-03-01 15:08:11 +01:00
06242d44fe [vimeo] add support for Vimeo Pro portfolio protected videos(closes #20070) 2019-03-01 08:14:34 +01:00
04c33bdfb3 release 2019.03.01 2019-03-01 01:03:51 +07:00
333f617b12 [ChangeLog] Actualize
[ci skip]
2019-03-01 01:02:36 +07:00
ff60ec8f02 [npo] Fix extraction (#20084) 2019-03-01 00:47:18 +07:00
9d9a8676dc [francetv:site] Extend video id regex (closes #20029, closes #20071) 2019-02-28 23:26:52 +07:00
db1c3a9d3f [periscope] Extract width and height (closes #20015) 2019-02-27 03:41:15 +07:00
55b8588f0e [servus] Fix extraction (closes #19297) 2019-02-24 23:20:06 +07:00
f0228f56fb [bbccouk] Make subtitles non fatal (#19651) 2019-02-24 21:01:25 +07:00
8c80603f1a [downloader/external] Add support for rate limit and retries for wget 2019-02-23 01:00:03 +07:00
37b239b3b6 [downloader/external] Fix infinite retries for curl (closes #19303) 2019-02-23 00:43:29 +07:00
caf48f557a [metacafe] Fix family filter bypass (closes #19287) 2019-02-21 05:59:07 +07:00
77a842c892 release 2019.02.18 2019-02-18 02:11:11 +07:00
c76fc5b22a [ChangeLog] Actualize
[ci skip]
2019-02-18 02:10:06 +07:00
388cfbd3d8 [tvp:website] Improve support 2019-02-17 14:27:00 +07:00
d93083789b [tvp:series] Fix extraction 2019-02-17 14:09:30 +07:00
34568dc296 [tvp] Detect unavailable videos 2019-02-17 13:39:00 +07:00
3c9647372e [tvp] Fix description extraction, make thumbnail optional and fix tests 2019-02-17 13:38:21 +07:00
659e93fcf5 [linuxacademy] Add extractor (closes #12207) 2019-02-17 07:12:10 +07:00
c9a0ea6e51 [bilibili] Update keys (closes #19233) 2019-02-17 05:00:16 +07:00
d7d513891b [udemy] Extend _VALID_URLs (closes #14330, closes #15883) 2019-02-17 01:05:01 +07:00
ae65c93a26 [udemy] Update User-Agent and detect captcha (closes #14713, closes #15839, closes #18126) 2019-02-17 00:58:13 +07:00
ba2e3730d1 [noovo] Fix extraction (closes #19230) 2019-02-16 22:45:53 +07:00
2b2da3ba10 [rai] Relax _VALID_URL (closes #19232) 2019-02-15 23:56:29 +07:00
794c1b6e02 [vshare] Pass Referer to download request (closes #19205, closes #19221) 2019-02-14 23:43:16 +07:00
7bee705d8f [openload] Add support for oload.live 2019-02-14 22:28:16 +07:00
6f5c1807f4 [imgur] Use video id as title fallback (closes #18590) 2019-02-13 00:02:29 +07:00
985637cbbf [twitch] Add new source format detection approach (closes #19193) 2019-02-12 00:13:50 +07:00
7d8b89163c [tvplayhome] Fix video id extraction (closes #19190) 2019-02-11 04:41:28 +07:00
d777f3e81c [tvplayhome] Fix episode metadata extraction (closes #19190) 2019-02-11 04:39:23 +07:00
4c0e0dc9dc [rutube:embed] Fix extraction and add support private videos (closes #19163) 2019-02-11 00:49:51 +07:00
f516f44094 [soundcloud] Extract more metadata 2019-02-10 23:44:08 +07:00
e9dee7f1b2 [trunews] Add extractor (closes #19153) 2019-02-09 23:50:27 +07:00
91effe22a0 [linkedin:learning] extract chapter_number and chapter_id(closes #19162) 2019-02-08 07:21:50 +01:00
04eacf5453 release 2019.02.08 2019-02-08 01:12:51 +07:00
f1f5b47255 [ChangeLog] Actualize
[ci skip]
2019-02-08 01:10:12 +07:00
1211bb6dac [YoutubeDL] Improve _make_archive_id (closes #19149) 2019-02-08 01:08:48 +07:00
4de3cb883c [malltv] Fix issues and simplify (closes #17856) 2019-02-08 00:43:31 +07:00
22f5f5c6fc [malltv] Add extractor (closes #18058) 2019-02-08 00:43:26 +07:00
49bd993fd9 [spankbang:playlist] Add extractor (closes #19145) 2019-02-08 00:09:50 +07:00
f06a1cabe8 [spankbang] Extend _VALID_URL 2019-02-08 00:07:29 +07:00
241c5d7d38 [trutv] fix extraction(closes #17336) 2019-02-06 19:38:10 +01:00
8fecc7353d [toutv] fix authentication(closes #16398)(closes #18700) 2019-02-06 13:59:33 +01:00
5dda1edef9 [pornhub] Improve and simplify (closes #19135) 2019-02-05 23:09:24 +07:00
d2d970d07e [pornhub] Fix tags and categories extraction (closes #13720) 2019-02-05 23:08:49 +07:00
48fb963b2f [pornhd] Fix formats extraction 2019-02-05 00:07:37 +07:00
70c3ee1367 [pornhd] Extract like count 2019-02-05 00:06:04 +07:00
07fbfef1c7 [radiocanada] switch to the new media requests(closes #19115) 2019-02-03 12:10:41 +01:00
eecf788b90 [teachable] add support for courses.workitdaily.com (closes #18871) 2019-02-03 09:10:35 +01:00
0efcb5a2fe [vporn] Remove extractor (closes #16276)
Handled by generic extractor
2019-02-03 00:33:45 +07:00
7c5307f4c4 [soundcloud:pagedplaylist] Improve (closes #19086) 2019-02-02 23:40:30 +07:00
6cc6e0c34d [soundcloud:pagedplaylist] Add ie and title to entries (#19022)
rel: https://github.com/rg3/youtube-dl/issues/19022
2019-02-02 23:40:22 +07:00
b9bc1cff72 [drtuber] Extract duration 2019-02-02 06:04:00 +07:00
e9fef7ee4e [YoutubeDL] Fallback to ie_key of matching extractor while making download archive id when no explicit ie_key is provided (#19022) 2019-02-02 05:44:31 +07:00
b6423e6ca2 [soundcloud:user] Update tests 2019-02-02 04:11:32 +07:00
3ef2da2d21 [soundcloud] Fix paged playlists extraction, add support for albums and update client id 2019-02-02 04:00:29 +07:00
49fe4175ae [drtv] Improve preference (closes #19079) 2019-02-01 01:49:33 +07:00
9613e14a92 [openload] Add support for openload.pw and oload.pw (closes #18930) 2019-01-31 00:15:45 +07:00
15e832ff2a [openload] Add support for oload.info 2019-01-30 23:39:02 +07:00
645c4885cf [crackle] authorize media detail request(closes #16931) 2019-01-30 14:44:23 +01:00
7b0f9df23d release 2019.01.30.1 2019-01-30 06:19:36 +07:00
c2a0fe2ea7 [ChangeLog] Actualize
[ci skip]
2019-01-30 06:17:25 +07:00
ce52c7c111 [postprocessor/ffmpeg] Fix avconv processing broken in #19025 (closes #19067) 2019-01-30 06:15:23 +07:00
1063b4c707 release 2019.01.30 2019-01-30 00:08:39 +07:00
ca01e5f903 [ChangeLog] Actualize
[ci skip]
2019-01-30 00:05:32 +07:00
5496754ae4 [fox] Remove unused imports 2019-01-30 00:03:19 +07:00
9868f1ab18 [yourporn] Improve (closes #19061) 2019-01-30 00:01:17 +07:00
41cff90c41 [yourporn] Fix extraction and extract duration (closes #18815, closes #18852)
change cdn to cdn4 for the video_url
2019-01-29 23:58:44 +07:00
a2d821d711 [drtv] Improve extraction (closes #19039)
+ Add support for EncryptedUri videos
+ Extract more metadata
* Fix subtitles extraction
2019-01-29 23:33:09 +07:00
6df196f32e [fox] add support for locked videos using cookies(closes #19060) 2019-01-29 00:31:49 +01:00
41c2c254d3 [fox] fix extraction for free videos(#19060) 2019-01-28 22:39:19 +01:00
a81daba231 [zattoo] Add support for tv.salt.ch 2019-01-29 02:20:46 +07:00
61ff92e11e [postprocessor/ffmpeg] Wrap loglevel args in encodeArgument 2019-01-29 01:59:56 +07:00
1397a790ff [postprocessor/ffmpeg] Disable "Last message repeated" messages which cause non-zero exit status (#19025) 2019-01-28 22:59:00 +07:00
7f903dd8bf [postprocessor/ffmpeg] Do not copy Apple TV chapter tracks while embedding subtitles (closes #19042)
Related issue: https://trac.ffmpeg.org/ticket/6016
2019-01-28 22:57:14 +07:00
2b3afe6b0f [postprocessor/ffmpeg] Disable "Last message repeated" messages which cause non-zero exit status 2019-01-27 22:24:37 -05:00
e71be6ee9f release 2019.01.27 2019-01-27 21:28:09 +07:00
bf8ebc9cfe [ChangeLog] Actualize
[ci skip]
2019-01-27 21:26:28 +07:00
1fcc91663b [vice] fix extraction for locked videos(closes #16248) 2019-01-27 10:53:38 +01:00
30cd1a5f39 [wakanim] detect DRM protected videos 2019-01-26 22:53:09 +01:00
458fd30f56 [extractor/common] Extract season in _json_ld 2019-01-27 04:36:58 +07:00
845333acf6 [wakanim] Add extractor (closes #14374) 2019-01-27 04:14:54 +07:00
252abb1e8b [README.md] Mention more convenience extraction functions 2019-01-26 15:29:19 +07:00
ae18d58297 [usatoday] fix extraction for videos with custom brightcove partner id(closes #18990) 2019-01-25 11:02:15 +01:00
1602a240a7 [drtv] Fix extraction (closes #18989) 2019-01-25 04:16:49 +07:00
0eba178fce [nhk] Extend _VALID_URL (closes #18968) 2019-01-25 04:04:58 +07:00
eb35b163ad [postprocessor/ffmpeg] fallback to ffmpeg/avconv for audio codec detection(closes #681) 2019-01-24 20:23:04 +01:00
118afcf52f [go] fix adobe pass requests for Disney Now(closes #18901) 2019-01-23 22:16:52 +01:00
9713d1d1e0 [openload] Add support for oload.club (closes #18969) 2019-01-24 02:30:12 +07:00
a1e171233d release 2019.01.24 2019-01-24 01:46:23 +07:00
7d311586ed [ChangeLog] Actualize
[ci skip]
2019-01-24 01:44:09 +07:00
e118a8794f [YoutubeDL] Fix typo in string negation implementation and add more tests (closes #18961) 2019-01-24 01:39:39 +07:00
435e382423 release 2019.01.23 2019-01-23 04:46:55 +07:00
0670bdd8f2 [ChangeLog] Actualize
[ci skip]
2019-01-23 04:43:55 +07:00
71a1f61700 [pornhub] Apply scrape detection bypass for all extractors 2019-01-23 04:12:06 +07:00
6510a3aa97 [crunchyroll] Extend _VALID_URL (closes #18955) 2019-01-23 03:55:41 +07:00
278d061a0c [pornhub] Bypass scrape detection (closes #5930) 2019-01-23 03:51:29 +07:00
503b604a31 [vrv] fix oauth signing for python 2(#14307) 2019-01-22 18:21:37 +01:00
4b85f0f9db [vrv] add support for authentication(closes #14307) 2019-01-22 14:38:40 +01:00
19d6991312 [videomore] Improve extraction and fix season extractor (closes #18908) 2019-01-22 03:03:53 +07:00
07f9febc4b [tnaflix] Pass Referer in metadata request (closes #18925) 2019-01-20 22:07:01 +07:00
fad4ceb534 [utils] Fix urljoin for paths with non-http(s) schemes 2019-01-20 20:22:19 +07:00
6945b9e78f [extractor/common] improve jwplayer relative url handling(closes #18892) 2019-01-20 13:31:52 +01:00
29cfcb43da [radiocanada] Relax DRM check 2019-01-20 18:33:09 +07:00
a1a4607598 [vimeo] Fix video password verification for videos protected by Referer HTTP header 2019-01-20 18:21:31 +07:00
73c19aaa9f [hketv] Improve and simplify (closes #18696) 2019-01-20 17:43:11 +07:00
289ef490f7 [hketv] Add extractor 2019-01-20 17:43:11 +07:00
6ca3fa898c [streamango] Add support for fruithosts.net 2019-01-20 16:24:21 +07:00
31fbedc06a [instagram] Add base extractor for playlists and tag extractor 2019-01-20 16:10:46 +07:00
15870747f0 [odnoklassniki] Detect paid videos 2019-01-20 15:15:01 +07:00
fc746c3fdd [test/test_InfoExtractor] add test for #18923 2019-01-20 09:05:12 +01:00
4e58d9fabb [README.md] Fix formatting 2019-01-20 14:23:35 +07:00
2cc779f497 [YoutubeDL] Add negation support for string comparisons in format selection expressions (closes #18600, closes #18805) 2019-01-20 13:48:49 +07:00
379306ef55 [extractor/common] fix typo 2019-01-19 21:35:02 +01:00
f28363ad1f [ted] correct acodec for http formats(#18923) 2019-01-19 21:25:53 +01:00
2bfc1d9d68 [extractor/common] imporove HLS video only format detection(closes #18923) 2019-01-19 21:25:15 +01:00
e2dd132f05 [cartoonnetwork] fix extraction(closes #15664)(closes #17224) 2019-01-17 23:56:37 +01:00
79fec976b0 [vimeo] fix extraction for password protected player URLs(closes #18889) 2019-01-17 09:44:29 +01:00
29639b363d release 2019.01.17 2019-01-17 10:27:17 +07:00
f53cecd796 [ChangeLog] Actualize
[ci skip]
2019-01-17 10:25:50 +07:00
fa4ac365f6 [youtube] Extend JS player signature function name regexes (closes #18890, closes #18891, closes #18893) 2019-01-17 10:24:44 +07:00
bfc8eeea57 release 2019.01.16 2019-01-16 02:24:08 +07:00
b0d73a7456 [ChangeLog] Actualize
[ci skip]
2019-01-16 02:20:10 +07:00
4fe54c128a [youtube] Update tests and add a tests for #18804 2019-01-16 02:18:27 +07:00
a16c7c033a [test/helper] Add support for maxcount and count collection len test checkers 2019-01-16 02:17:49 +07:00
2f483bc1c3 [youtube] Skip unsupported adaptive stream type (#18804) 2019-01-16 01:28:50 +07:00
561b456e2d [youtube] Extract DASH formats from player response (closes #18804) 2019-01-16 01:12:58 +07:00
929ba3997b [funimation] fix extraction(closes #14089) 2019-01-15 10:25:31 +01:00
10026329c2 [skylinewebcams] Fix extraction (closes #18853) 2019-01-14 23:23:51 +07:00
3b983ee471 [curiositystream] add support for non app urls 2019-01-13 15:46:54 +01:00
f1ab3b7de7 [downloader/hls] fix uplynk ad skipping(closes #18824) 2019-01-13 10:03:27 +01:00
d65f6e734b [bitchute] Check formats (#18833) 2019-01-13 03:59:38 +07:00
ed8db0a25c [wistia] Extend _VALID_URL (closes #18823) 2019-01-12 04:58:01 +07:00
60a899bb7e [README.md] Fix typo 2019-01-12 02:15:48 +07:00
cbdc688c41 [postprocessor/ffmpeg] Relax ubuntu ffmpeg version regex 2019-01-12 00:30:06 +07:00
5caa531a1a [postprocessor/ffmpeg] PEP 8 2019-01-11 23:47:23 +07:00
a64646e417 [postprocessor/ffmpeg] sanitize ffmpeg version for Ubuntu and Arch Linux systems(closes #18813) 2019-01-11 15:12:10 +01:00
c469e8808c [playplustv] add support for playplus.com(#18789) 2019-01-10 18:49:13 +01:00
b64f6e690f release 2019.01.10 2019-01-10 23:26:54 +07:00
a4491dd55c [ChangeLog] Actualize
[ci skip]
2019-01-10 23:23:19 +07:00
c3e543893b [youtube] Extract live HLS URL from player response (closes #18799) 2019-01-10 22:46:53 +07:00
432aba1c5e [outsidetv] Add new extractor(closes #18774) 2019-01-10 10:54:46 +01:00
7c072f00d6 [jwplatform] use JW Platform Delivery API V2 and add support for more urls 2019-01-10 10:50:18 +01:00
96c186e1fd [fox] add support National Geographic(closes #17985)(closes #15333)(closes #14698) 2019-01-10 09:05:00 +01:00
4ad159c7b0 [playplustv] Add new extractor(closes #18789) 2019-01-09 20:39:48 +01:00
65615be368 [globo] set GLBID cookie manually(closes #17346) 2019-01-09 19:17:58 +01:00
3c1089dba4 [gaia] Add new extractor(#14605) 2019-01-09 14:23:26 +01:00
6089ff40e7 [youporn] Fix title and description extraction (closes #18748) 2019-01-09 00:37:01 +07:00
2543938bbe [hungama] Add support for videos (closes #17402) 2019-01-08 10:03:44 +07:00
440863ade1 [extractor/common] Use episode name as title in _json_ld 2019-01-08 10:02:49 +07:00
391256dc0e [extractor/common] Add support for movies in _json_ld 2019-01-08 10:02:00 +07:00
06b4b90c70 [hungama] Fix code and extract more metadata (closes #18771) 2019-01-08 09:14:47 +07:00
8cb5c2181a [hungama] Add extractor 2019-01-08 09:14:22 +07:00
0266854f63 [dtube] Update test 2019-01-08 08:46:34 +07:00
bcc334a3c6 [dtube] Fix extraction (closes #18741) 2019-01-08 08:44:42 +07:00
e9a50fba86 [utils] Fix typo 2019-01-07 01:02:34 +07:00
04fb6928da [postprocessor/ffmpeg] Embed subtitles with non-standard language codes (refs #18765) 2019-01-07 00:57:55 +07:00
b7acc83550 [utils] Add language codes replaced in 1989 revision of ISO 639 to ISO639Utils (closes #18765) 2019-01-07 00:57:48 +07:00
de0359c0af [tvnow] Fix and rework extractors, prepare for a switch to the new API (closes #17245, closes #18499) 2019-01-05 03:44:19 +07:00
c87f65e43d [carambatv:page] Fix extraction (closes #18739) 2019-01-04 22:21:53 +07:00
d7c3af7a72 release 2019.01.02 2019-01-02 23:52:54 +07:00
aeb72b3a41 [ChangeLog] Actualize
[ci skip]
2019-01-02 23:51:23 +07:00
2122d7151d [discovery] Use geo verification headers 2019-01-02 23:46:07 +07:00
751e051557 [packtpub] Add support for subscription.packtpub.com (closes #18718) 2019-01-02 17:26:15 +07:00
d226c560a6 Refactor code to use url_result 2019-01-01 23:56:05 +07:00
8437f5089f [README.md] Add long lines policy to coding conventions 2019-01-01 23:50:02 +07:00
1d803085d7 [yourporn] Fix extraction 2019-01-01 23:26:59 +07:00
696f4e4114 [README.md] Add more guide lines for regular expressions 2019-01-01 23:13:39 +07:00
0e713dbb11 [acast:channel] Add support for play.acast.com (closes #18587) 2019-01-01 22:48:06 +07:00
9b5c8751ee [extractors] Add missing age limits 2019-01-01 21:39:18 +07:00
d9f1123c08 [rmcdecouverte] Improve, bypass geo restriction and add support for live (closes #18697) 2019-01-01 20:51:19 +07:00
3d8eb6beb9 [rmcdecouverte] Update _VALID_URL (closes #18595) 2019-01-01 20:50:57 +07:00
38d15ba7f9 [manyvids] Fix extraction (closes #18604, closes #18614) 2019-01-01 20:31:48 +07:00
6b688b8942 [bitchute] Fix extraction (closes #18567) 2019-01-01 18:12:44 +07:00
9d9daed464 release 2018.12.31 2018-12-31 23:59:52 +07:00
32ac3d49ae [ChangeLog] Actualize
[ci skip]
2018-12-31 23:57:46 +07:00
373941c5f0 [bbc] Add support for another embed pattern (closes #18643) 2018-12-31 23:20:40 +07:00
4e1ddc8da9 [npo:live] Add support for npostart.nl (closes #18644) 2018-12-31 21:05:07 +07:00
e4d51e751e [beeg] Fix extraction (closes #18610, closes #18626) 2018-12-31 20:59:53 +07:00
c2dd2dc086 [youtube] Unescape HTML for series (closes #18641) 2018-12-31 19:57:01 +07:00
140a13f5de [youtube] extract more format metadata 2018-12-26 10:56:10 +01:00
825cd268a3 [youtube] detect DRM protected videos(#1774) 2018-12-26 10:56:10 +01:00
63529e935c [youtube] Relax html5 player regexes (closes #18465, closes #18466) 2018-12-23 16:57:10 +07:00
4273caf5c7 [youtube] Extend html5 player regex (closes #17516) 2018-12-23 16:40:16 +07:00
e1a0628797 [liveleak] add support for another embed type and restore original format extraction 2018-12-20 23:22:51 +01:00
835e45abab [crackle] extract ism and http formats 2018-12-19 22:08:01 +01:00
904bb599be [README.md] Add flake8 instructions 2018-12-20 03:22:10 +07:00
65e29cdac3 [twitter] pass referer with card request(closes #18579) 2018-12-18 22:49:01 +01:00
4ee1845454 [mediasite] Extend _VALID_URL even more 2018-12-18 01:55:13 +07:00
cfd13c4c45 [mediasite] Relax _VALID_URL 2018-12-18 00:03:00 +07:00
386d1fea79 [lecturio] Add support for lecturio.de (closes #18562) 2018-12-17 23:43:12 +07:00
7216e9bff7 [discovery] Add support for Scripps Networks watch domains(closes #17947) 2018-12-17 16:35:29 +01:00
4cee62ade0 release 2018.12.17 2018-12-17 05:37:50 +07:00
cbb3e4b14f [ChangeLog] Actualize
[ci skip]
2018-12-17 05:34:55 +07:00
752582183a [ard:beta] Improve extraction robustness, fix subtitles extraction, improve geo restricted videos extraction 2018-12-17 05:29:59 +07:00
1c82122741 [ard:beta] Relax _VALID_URL (closes #18441) 2018-12-17 04:51:57 +07:00
50a498a68e [acast] Extend _VALID_URL 2018-12-17 04:32:59 +07:00
252e172dea [acast] Add support for embed.acast.com 2018-12-17 04:29:12 +07:00
90046d7761 [iprima] Relax _VALID_URL (closes #18515) 2018-12-17 04:10:36 +07:00
c8b3751086 [vrv] fix initial state extraction 2018-12-16 14:29:18 +01:00
21c340b83f [youtube] Fix mark watched (closes #18546) 2018-12-16 19:40:49 +07:00
c984196cf1 [README.md] Bind info dict URLs to a fixed blob (closes #18492) 2018-12-15 23:59:17 +07:00
7f41a598b3 [safari] Add support for learning.oreilly.com (closes #18510) 2018-12-15 23:08:14 +07:00
8fe104947d [youtube] Fix multifeed extraction (closes #18531) 2018-12-15 22:25:12 +07:00
0a05cfabb6 [lecturio] Improve subtitles extraction (closes #18488) 2018-12-11 23:45:02 +07:00
13e17cd28e [uol] fix format url extraction(closes 18480) 2018-12-10 15:02:54 +01:00
102a4e54c5 [teachable] Remove debug output 2018-12-10 10:10:28 +07:00
6e29458f24 [test/testdata/cookies/session_cookies.txt] Fix empty expires test data 2018-12-10 04:30:00 +07:00
59c3940165 [ard:mediathek] Add support for classic.ardmediathek.de (closes #18473) 2018-12-10 01:37:10 +07:00
cefe42c412 release 2018.12.09 2018-12-09 23:11:32 +07:00
24cc64254c [ChangeLog] Actualize
[ci skip]
2018-12-09 23:08:16 +07:00
9e02c2c704 [YoutubeDLCookieJar] Add test for keeping session cookies 2018-12-09 22:57:00 +07:00
5ee7ae5c75 [teachable] Add support for teachable based platform sites (closes #5451, closes #18150, closes #18272) 2018-12-09 22:36:36 +07:00
3ad6dabd33 [aenetworks] add support for History Vault(closes #18460) 2018-12-09 10:04:00 +01:00
5f47a60c5d [imgur] improve gallery and album detection and extraction(closes #9133)(closes #16577)(closes #17223)(closes #18404) 2018-12-09 09:35:17 +01:00
1bab343704 [YoutubeDL] Introduce YoutubeDLCookieJar and clarify the rationale behind session cookies (closes #12929) 2018-12-09 06:47:49 +07:00
1d88b3e6e6 [YoutubeDL] Recognize expires=0 as session cookies and send session cookies with requests 2018-12-09 06:05:37 +07:00
9235b5091c [iprima] Relax _VALID_URL (closes #18453) 2018-12-09 00:01:11 +07:00
c3c098dcf2 [hotstar] fix video data extraction(closes #18386) 2018-12-07 18:52:01 +01:00
8c5879715f [ard:mediathek] Fix title and description extraction (closes #18349) 2018-12-07 03:41:02 +07:00
ebb0449049 [xvideos] Switch to HTTPS (closes #18422) 2018-12-07 03:36:08 +07:00
dfe0a3a9d2 [lecturio] Add extractor (closes #18405) 2018-12-07 03:27:11 +07:00
c976873c5b [nrktv:series] Add support for extra materials 2018-12-07 00:54:58 +07:00
15699ec8b0 [nrktv:season,series] Fix extraction and update tests (closes #17159, closes #17258) 2018-12-07 00:49:24 +07:00
33cc1ea586 [nrktv] Relax _VALID_URL (closes #18304, closes #18387) 2018-12-07 00:00:06 +07:00
ae9d77dab5 [yourporn] Fix extraction (closes #18424) 2018-12-06 23:24:35 +07:00
8bb0c9cc16 [tbs] fix info extraction(fixes #18403) 2018-12-05 07:03:00 +01:00
5547014ad9 [gamespot] add support reviews URLs 2018-12-02 20:01:56 +01:00
ab896fa894 release 2018.12.03 2018-12-03 00:10:20 +07:00
1fa59a928e [ChangeLog] Actualize
[ci skip]
2018-12-03 00:06:54 +07:00
ce18a19be9 [tiktok] Improve extraction and add support for user pages (closes #18135) 2018-12-02 02:42:56 +07:00
1ead840d2c [tiktok] Add extractor (closes #18108) 2018-12-02 02:42:56 +07:00
aa374bc78e [utils] Fix random_birthday to generate existing dates only 2018-12-02 00:05:15 +07:00
3430ff9b07 [pornhub] Use actual URL host for requests (closes #18359) 2018-12-01 16:45:51 +07:00
f012823082 [lynda] Fix authentication (closes #18158) 2018-11-30 01:20:27 +07:00
16597c2f94 [gfycat] Update API endpoint (closes #18333) 2018-11-30 01:07:07 +07:00
adbbdefc81 [hotstar] Add support for alternative app state layout (closes #18320) 2018-11-30 00:48:15 +07:00
053e5b12b2 [azmedien] Fix extraction (closes #18334) 2018-11-30 00:12:18 +07:00
d9df8f120b [vimeo] extract VHX subtitles 2018-11-28 20:13:36 +01:00
ca01d17884 [vimeo] Add support for VHX(Vimeo OTT)(#14835) 2018-11-28 19:53:45 +01:00
d19600df07 [joj] Fix extraction (closes #18280) 2018-11-24 22:14:27 +07:00
641e86e3cf [wistia] Add support for fast.wistia.com (closes #18287) 2018-11-24 21:47:41 +07:00
6864855eb1 [tests] Fix invalid escape sequences 2018-11-23 00:43:42 +07:00
d861a9d581 release 2018.11.23 2018-11-23 00:16:45 +07:00
66173211c4 [ChangeLog] Actualize
[ci skip]
2018-11-23 00:14:43 +07:00
6f2883a2df [mixcloud] base64 decode before decryption 2018-11-21 23:25:38 +01:00
560020da30 [mixcloud] fallback to hardcoded decryption key(closes #18016) 2018-11-21 23:21:05 +01:00
305ce767d5 [travis] Add python 3.8-dev build 2018-11-22 02:34:35 +07:00
157eef3e63 [setup.py] Add python 3.8 classifier 2018-11-22 02:08:41 +07:00
bd2d553c7b [travis] Add python 3.7 build 2018-11-22 02:01:39 +07:00
af60e81e3c [setup.py] Add more relevant classifiers 2018-11-22 02:01:39 +07:00
a843464a7e [nbc] fix NBCNews article extraction(closes #16194) 2018-11-21 12:10:06 +01:00
6866f24494 [foxsports] update test 2018-11-21 12:08:46 +01:00
4e33e0792a [loc] update test 2018-11-21 12:00:50 +01:00
35328915b5 [foxsports] fix extraction(closes #17543) 2018-11-21 09:46:36 +01:00
6c882aa899 [loc] relax _VALID_URL regex and improve formats extraction 2018-11-21 09:46:36 +01:00
183417a50f [ciscolive:search] Add support for pagination 2018-11-21 06:10:43 +07:00
6a6d7f0641 [ciscolive] Fix issues and improve extraction (closes #17984) 2018-11-21 06:10:39 +07:00
05bd5e9c77 [ciscolive] Add extractor 2018-11-21 06:10:30 +07:00
15ed5a2784 [nzz] Relax kaltura regex 2018-11-21 02:50:40 +07:00
2e1280ed43 [sixplay] fix format extraction 2018-11-19 18:15:51 +01:00
8578ea4dcb [bitchute] use _html_search_regex for title extraction 2018-11-18 16:15:27 +01:00
9b27a78a88 [kaltura] limit requested MediaEntry fields 2018-11-18 16:15:27 +01:00
964b989dc8 [americastestkitchen] Add support for zype embeds (closes #18225) 2018-11-18 20:45:25 +07:00
f97c099131 [pornhub] Move test to correct place 2018-11-18 11:14:46 +07:00
1febf99da1 [pornhub] Add pornhub.net alias 2018-11-18 06:26:08 +07:00
4167148fa4 [nova:embed] Fix extraction (closes #18222) 2018-11-18 01:11:10 +07:00
5bb0479269 release 2018.11.18 2018-11-18 00:11:54 +07:00
02df855e13 [ChangeLog] Actualize
[ci skip]
2018-11-18 00:07:40 +07:00
006374e3ae [wwe] Fix issues, extract subtitles and add support for playlists (closes #14781, closes #17450) 2018-11-17 23:59:52 +07:00
11d19ff503 [wwe] Add extractor 2018-11-17 23:59:46 +07:00
a640c4d226 [vk] Detect geo restriction 2018-11-17 20:59:13 +07:00
d0058c76d5 [openload] Use original host during extraction (closes #18211) 2018-11-17 16:59:20 +07:00
0919cd4d01 [atvat] Fix extraction (closes #18041) 2018-11-17 00:18:50 +07:00
2599956c9f [rte] Add support for new API endpoint (closes #18206) 2018-11-17 00:07:59 +07:00
9b9b3501c5 [tnaflixnetwork:embed] Fix extraction (closes #18205) 2018-11-16 22:55:35 +07:00
730c0d12a0 [picarto] Extract more metadata (closes #16518) 2018-11-11 16:17:59 +07:00
f17a24a6df [picarto] Use API and add token support
This is just more reliable than trying to extract
it from the page itself.
2018-11-11 16:17:56 +07:00
83852e57bf [zype] Add extractor (closes #18143) 2018-11-11 00:44:49 +07:00
96a91b1551 [vivo] Fix extraction (closes #18139) 2018-11-10 23:37:27 +07:00
cab26223bf [ruutu] Update API endpoint (closes #18138) 2018-11-10 15:26:04 +07:00
532782ade1 release 2018.11.07 2018-11-07 01:38:25 +07:00
f81d44aab6 [ChangeLog] Actualize
[ci skip]
2018-11-07 09:58:08 +07:00
2511eee215 [youtube] Add another JS signature function name regex (closes #18091, closes #18093, closes #18094) 2018-11-07 09:55:59 +07:00
0df514f07e [facebook] fix tahoe request(closes #17171) 2018-11-06 21:22:27 +01:00
432cd48410 [cliphinter] Fix extraction (closes #18083) 2018-11-06 23:29:42 +07:00
c0345b825f [youtube:playlist] Add support for invidio.us (closes #18077) 2018-11-05 19:08:39 +07:00
2004e2210b [osnateltv] Update host 2018-11-05 17:09:57 +07:00
16d896b2a7 [zattoo] Arrange API hosts for derived extractors (closes #18035) 2018-11-05 15:52:46 +07:00
22e07ce502 [README.md] Improve documentation on safe metadata extraction and add more examples 2018-11-05 00:11:36 +07:00
dbdaaa231a [youtube] Add fallback metadata extraction from videoDetails (closes #18052) 2018-11-03 06:26:16 +07:00
38c32dbf19 release 2018.11.03 2018-11-03 02:57:48 +07:00
a085410936 [ChangeLog] Actualize
[ci skip]
2018-11-03 02:56:14 +07:00
6895ea4d3f [laola1tv:embed] Set correct stream access URL scheme (closes #16341) 2018-11-03 02:45:37 +07:00
faac1c1f70 [ehftv] Add extractor (closes #15408) 2018-11-03 02:45:32 +07:00
573531dcfb [azmedien] Simplify (closes #17746) 2018-11-03 01:33:42 +07:00
da56fb631f [azmedien] Adopt to major site redesign (closes #17745) 2018-11-03 01:33:36 +07:00
95e42d7336 [extractor/common] Ensure response handle is not prematurely closed before it can be read if it matches expected_status (resolves #17195, closes #17846, resolves #17447) 2018-11-03 01:18:20 +07:00
cf0db4d997 [twitcasting] Improve extraction and fix issues (closes #17981) 2018-11-03 00:28:25 +07:00
036f905161 [twitcasting] Add extractor 2018-11-03 00:28:21 +07:00
4b6aca17cc [orf:tvthek] Improve extraction and remove unused code (closes #17956, closes #18024) 2018-11-02 23:47:23 +07:00
c620694c97 [orf:tvthek] Fix extraction (closes #17737)
use _extract_m3u8_formats and _extract_f4m_formats helper functions
closes #17737
2018-11-02 23:47:17 +07:00
061ea3a776 [openload] Add support for oload.fun 2018-11-02 23:08:41 +07:00
c70ba664f1 [njpwworld] Fix authentication (closes #17427) 2018-11-01 01:40:02 +07:00
f16679e843 [cnbc:video] Fix _VALID_URL (#17110) 2018-10-30 04:57:51 +07:00
b14475724b [linkedin:learning:course] use url_transparent type for playlist entries 2018-10-29 21:49:12 +01:00
aa7e974a2a [linkedin:learning] Add new extractor(closes #13545) 2018-10-29 19:28:29 +01:00
9aac22c195 [theplatform] Improve error detection (#13222) 2018-10-30 00:22:44 +07:00
94db1f7f3b [cnbc] Simplify extraction (closes #14280, closes #17110) 2018-10-29 23:55:55 +07:00
ffa7b2bfee [cbnc] Add support for new URL schema (closes #14193) 2018-10-29 23:54:14 +07:00
2943397e87 [aparat] Improve extraction and extract more metadata (closes #17445, closes #18008) 2018-10-29 23:32:45 +07:00
9c4a83a1be [aparat] Fix extraction 2018-10-29 23:31:24 +07:00
9ff558f67f release 2018.10.29 2018-10-29 00:39:29 +07:00
c2fe21efaa [ChangeLog] Actualize
[ci skip]
2018-10-29 00:38:06 +07:00
476cf548e1 [sportbox] Improve extraction, add support for matchtv.ru and fix video id (closes #17978) 2018-10-29 00:21:50 +07:00
bebef10909 [extractor/common] Add validation for JSON-LD URLs 2018-10-29 00:21:45 +07:00
4c237ab787 [sportbox] Fix extraction 2018-10-29 00:21:40 +07:00
a1d1c63678 [screencast] Improve extraction (closes #14617, closes #17990) 2018-10-28 23:26:30 +07:00
1fafb32984 [screencast] Fix extraction (closes #14590) 2018-10-28 23:26:30 +07:00
c901cc38e5 [openload] Add support for oload.icu 2018-10-28 22:51:29 +07:00
022218f2f0 [ivi] Add support for ivi.tv 2018-10-28 22:49:10 +07:00
08c7d3dade [crunchyroll] Improve extraction failsafeness (closes #17991) 2018-10-28 22:12:54 +07:00
5e733b066a [dailymail] fix format extraction(closes #17976) 2018-10-26 05:41:57 +01:00
7d9e858132 [viewster] reduce format requests 2018-10-26 05:40:49 +01:00
b99b0bcfa0 [cwtv] handle api errors(closes #17905) 2018-10-17 06:22:56 +01:00
baeabf7742 [rutube] Use geo verification headers (closes #17897) 2018-10-16 23:19:44 +07:00
582797d780 [brightcove] remove unused variable 2018-10-15 20:47:12 +01:00
160c2773f6 [brightcove:legacy] add another fall back to brightcove:new 2018-10-15 18:41:57 +01:00
ee5fe42e44 [brightcove:legacy] fall back to brightcove:new(#13912) 2018-10-15 17:54:38 +01:00
f0ee386851 [tv3] remove extractor(closes #10461)(closes #15339) 2018-10-15 16:26:29 +01:00
a94e7c195e [ted] fix extraction for http and rtmp formats(closes #5941)(closes #17572)(closes #17894) 2018-10-15 11:52:01 +01:00
5d90a8a5f3 [openload] Add support for oload.cc 2018-10-07 20:05:45 +07:00
19a352854f [patreon] extract post_file url(#17792) 2018-10-05 22:45:04 +01:00
c9d891f19a [patreon] fix extraction(closes #14502)(closes #10471) 2018-10-05 20:11:01 +01:00
d96f976b0c release 2018.10.05 2018-10-05 02:31:30 +07:00
2e7ed29e34 [ChangeLog] Actualize
[ci skip]
2018-10-05 02:29:52 +07:00
21c1a00dd7 [pluralsight] Improve authentication (closes #17762) 2018-10-05 02:27:14 +07:00
0082f44a08 [dailymotion] Improve metadata extraction (closes #17706) 2018-10-05 02:03:25 +07:00
f60b9803a4 [dailymotion] Fix extraction (closes #17699) 2018-10-05 02:03:16 +07:00
d98cb62e55 [crunchyroll] switch to HTTPS for RpcApi(closes #17749) 2018-10-02 19:43:06 +01:00
05e7c184da [hotstar] fix extraction in python 2(closes #17696) 2018-10-02 06:07:43 +01:00
66d106f270 [philharmoniedeparis] Fix extraction and add support for pad.philharmoniedeparis.fr (closes #17705) 2018-10-01 23:29:49 +07:00
3c7da54c92 [jamendo] Add support for licensing.jamendo.com (closes #17724) 2018-10-01 22:05:18 +07:00
9795d93316 [openload] Add support for oload.cloud (closes #17710) 2018-10-01 21:48:59 +07:00
365343131d [pluralsight] Fix subtitles extraction (closes #17726, closes #17728) 2018-10-01 21:45:24 +07:00
85fa80d5f9 [vimeo] Add another config regex (closes #17690) 2018-10-01 21:13:43 +07:00
245cbb33bc [spike] fix Paramount Network extraction(closes #17677) 2018-09-28 15:13:25 +01:00
85cd69adcb [hotstar] fix extraction(closes #14694)(closes #14931)(closes #17637) 2018-09-26 08:13:36 +01:00
4c89a675dd release 2018.09.26 2018-09-26 11:58:25 +07:00
3d3499742c [ChangeLog] Actualize
[ci skip]
2018-09-26 11:56:15 +07:00
c17e100b96 [pluralsight] Fix subtitles extraction (closes #17671) 2018-09-26 09:27:40 +07:00
8fd12a0831 [mediaset] Improve embed support (closes #17668) 2018-09-26 05:38:41 +07:00
60ce0c67fd [README.md] Document channel meta fields for output template 2018-09-25 23:43:41 +07:00
cd5a74a28e [youtube] Add support for invidio.us (closes #17613) 2018-09-24 00:14:49 +07:00
f6d7f7b474 [zattoo] Add support for more zattoo platform sites 2018-09-23 23:35:37 +07:00
21160a1792 [zattoo] Fix extraction (closes #17175) 2018-09-23 21:34:47 +07:00
4ac73fc170 [popcorntv] Remove debug output 2018-09-19 22:16:43 +07:00
28fcb7b061 [raiplay:playlist] Remove a debug leftover print() 2018-09-19 09:48:39 +07:00
3a9c928426 release 2018.09.18 2018-09-18 01:46:36 +07:00
d9b1cec171 [ChangeLog] Actualize
[ci skip]
2018-09-18 01:44:55 +07:00
e504b09070 [adobepass] Don't pollute default headers dict 2018-09-17 22:15:27 +07:00
c8f6ab8c38 [udemy] Don't pollute default headers dict 2018-09-17 22:14:53 +07:00
e2f61598be [twitch] Don't pollute default headers dict 2018-09-17 22:14:28 +07:00
c11485162b [youtube] Don't pollute default query dict (closes #17593) 2018-09-17 22:13:39 +07:00
1084563eaa [crunchyroll] Prefer hardsubless formats and formats in locale language 2018-09-16 23:54:58 +07:00
d9b0d118ad [vrv] Make format ids deterministic 2018-09-16 23:54:58 +07:00
8b40c92724 [vimeo] redirect to feature url only in the case of a trailer(closes #14591) 2018-09-15 06:31:15 +01:00
3661ebf2b6 [pornhub] Extract upload date (closes #17574) 2018-09-15 02:04:43 +07:00
0e7b8d3eac [extractor/common] Fix typos 2018-09-15 01:53:01 +07:00
127103b643 [porntube] Extract channel meta fields 2018-09-15 01:25:32 +07:00
d03beddf0f [vimeo] Extract channel meta fields 2018-09-15 01:24:48 +07:00
dd4c449219 [youtube] Extract channel meta fields (closes #9676, closes #12939) 2018-09-15 01:24:26 +07:00
6f1f59f39c [extractor/common] Introduce channel meta fields 2018-09-15 01:23:36 +07:00
15bf2ca0da [porntube] Fix extraction (closes #17541) 2018-09-15 00:50:36 +07:00
0f2aa0dcaa [asiancrush] Fix extraction (closes #15630) 2018-09-14 23:56:03 +07:00
db348e8849 [twitch:clips] Extend _VALID_URL (closes #17559) 2018-09-14 23:26:38 +07:00
f5b0175349 [vzaar] Add support for HLS 2018-09-11 02:41:05 +07:00
79facb2773 [tube8] Fix metadata extraction (closes #17520) 2018-09-11 02:32:01 +07:00
96dbf70de6 [eporner] Extract JSON-LD (closes #17519) 2018-09-11 02:32:01 +07:00
8476b4fd91 release 2018.09.10 2018-09-10 02:48:37 +07:00
14f577e31c [ChangeLog] Actualize
[ci skip]
2018-09-10 02:45:44 +07:00
25d110be30 [utils] Properly recognize AV1 codec (closes #17506) 2018-09-10 02:37:22 +07:00
a2637a2dda [iprima] Add support for prima.iprima.cz (closes #17514) 2018-09-10 01:34:49 +07:00
2e4350eec6 [generic] PEP 8 (#17455) 2018-09-08 17:24:59 +07:00
2c9d3b9962 [seznamzpravy] PEP 8 (#17455) 2018-09-08 17:24:48 +07:00
13ef64fd93 [motherless] PEP 8 (#17455) 2018-09-08 17:24:34 +07:00
6f9f3340bb [dtube] PEP 8 (#17455) 2018-09-08 17:24:09 +07:00
ae2384ff5f [tele5] Improve extraction (closes #7805, closes #7922, closes #17331, closes #17414) 2018-09-08 16:08:48 +07:00
d0de6a287a [tele5] Add extractor 2018-09-08 16:08:44 +07:00
d0c5fabc12 [nbc] Fix extraction of percent encoded URLs (closes #17374) 2018-09-08 14:44:06 +07:00
ad98d2eb74 release 2018.09.08 2018-09-08 03:42:28 +07:00
a41a506077 [ChangeLog] Actualize
[ci skip]
2018-09-08 03:40:06 +07:00
9a47fa35dd [youtube] Fix extraction (closes #17457, closes #17464) 2018-09-08 03:36:10 +07:00
2d4fe594c6 [pornhub:uservideos] Add support for new URLs (closes #17388) 2018-09-06 00:51:20 +07:00
09322cccdb [iprima] Confirm adult check (closes #17437) 2018-09-06 00:22:53 +07:00
aa1d5eb905 [slideslive] make the check for video_service_name case-insensitive(closes #17429) 2018-09-04 10:37:51 +01:00
93284ff2ea [radiojavan] Improve extraction (closes #17151) 2018-09-03 02:53:51 +07:00
0a9a8118ce [radiojavan] Fix extraction 2018-09-03 02:53:41 +07:00
3d08f63dc5 [generic] Skip unsuccessful jwplayer extraction (closes #16735) 2018-09-03 02:15:17 +07:00
27d8e089a2 release 2018.09.01 2018-09-01 18:40:23 +07:00
7bbc1b189a [ChangeLog] Actualize
[ci skip]
2018-09-01 18:36:18 +07:00
0b87e88453 [ard] Add support for one.ard.de 2018-09-01 16:42:30 +07:00
4d59db5b90 [niconico] Fix extraction on python3 (closes #17393) 2018-09-01 16:04:45 +07:00
4627995882 [crunchyroll] limit VRVIE inheritance to CrunchyrollIE 2018-09-01 10:04:10 +01:00
7f2611cb5b [ard] extract f4m formats 2018-09-01 08:40:38 +01:00
54a5be4dba [crunchyroll] parse vilos media data(closes #17343) 2018-09-01 08:16:41 +01:00
ed6919e737 [ard] beta mediathek: make regexp for JSON more robust 2018-09-01 01:59:13 +02:00
2b83da2463 [ard] Better format handling
Skip f4m, doesn't work (yet); correctly extract m3u8, and prefer plain HTTP files.
2018-09-01 00:45:36 +02:00
c1a37eb24a [ard] Add support for Beta ARD Mediathek
Thanks to https://blog.fefe.de/?ts=a577685d for pointing out support is missing.
2018-09-01 00:18:17 +02:00
4991e16c2a [bandcamp] Extract more metadata (closes #13197) 2018-08-31 03:35:55 +07:00
14b7a24c19 [bandcamp] Extract track_number (closes #17266) 2018-08-31 02:32:35 +07:00
73f3bdbeb4 [internazionale] Fix extraction of non-available-abroad videos 2018-08-31 02:15:46 +07:00
9e21e6d96b [utils] Improve remote address skipping and add support for python 2.6 (closes #17362) 2018-08-29 01:18:03 +07:00
8959018a5f [utils] Skip remote IP addresses non matching to source address' IP version (closes #13422) 2018-08-29 01:17:53 +07:00
eebbce5656 release 2018.08.28 2018-08-28 03:10:09 +07:00
56213aff1d [ChangeLog] Actualize
[ci skip]
2018-08-28 03:07:18 +07:00
409b9324da [youtube:playlist] Add support for music album playlists (OLAK5uy_ prefix) (closes #17361) 2018-08-28 02:20:32 +07:00
02df41354c [bitchute] Fix extraction by pass custom User-Agent 2018-08-27 22:04:56 +07:00
dd88fd65a5 [webofstories:playlist] Fix extraction (closes #16914) 2018-08-26 21:42:30 +07:00
287cf7e443 [generic] Remove unused import 2018-08-26 20:09:02 +07:00
dac6f7654a [tvplayhome] Add extractor (closes #17344) 2018-08-26 20:08:55 +07:00
e0b6e98871 [generic] Allow relative src for videojs embeds (closes #17324) 2018-08-24 23:12:53 +07:00
beff09505c [xfileshare] Add support for vidto.se (closes #17317) 2018-08-24 04:00:35 +07:00
135e6a1c10 [vidzi] Add support for vidzi.nu (closes #17316) 2018-08-24 02:36:56 +07:00
c707d2067d [nova:embed] Add extractor (closes #17282) 2018-08-22 23:18:07 +07:00
4c86163b60 release 2018.08.22 2018-08-22 02:32:18 +07:00
b662273989 [ChangeLog] Actualize
[ci skip]
2018-08-22 02:28:25 +07:00
df4d817bc3 [kinopoisk] Add extractor (closes #17283) 2018-08-22 02:19:30 +07:00
db192b2932 [yourporn] Add extractor (closes #17298) 2018-08-22 01:44:22 +07:00
52007de8ca [go] Add support for disneynow.go.com (closes #16299, closes #17264) 2018-08-22 01:14:47 +07:00
28f96cf407 [6play] Add support for play.rtl.hr 2018-08-22 00:06:27 +07:00
eda86b4335 [anvato] Fallback to generic API key for access key to API key lookup (closes #16788, closes #17254) 2018-08-21 23:45:18 +07:00
bf1245d236 [lci] Fix extraction (closes #17274) 2018-08-20 02:15:48 +07:00
6f356cbbcf [bbccouk] Extend _ID_REGEX (closes #17270) 2018-08-20 02:05:07 +07:00
0a74b45191 [cwtv] fix extraction(closes #17256) 2018-08-17 11:59:49 +01:00
d6ef8b4dd4 [nova] Fix extraction (closes #17241) 2018-08-16 00:11:41 +07:00
60c0856223 [utils] Use pure browser header for User-Agent (closes #17236) 2018-08-14 23:27:12 +07:00
57c68ec4c3 [generic] Add support for expressen embeds 2018-08-14 22:51:44 +07:00
24e0cd709f [raywenderlich] Adapt to site redesign (closes #17225) 2018-08-13 00:15:59 +07:00
4779420ce8 [redbulltv] add support redbull.com tv URLs(closes #17218) 2018-08-12 05:31:34 +01:00
de4c41b437 [bitchute] Improve page offset 2018-08-12 01:52:50 +07:00
b65e3b0636 [bitchute] Add extractor (closes #14052) 2018-08-12 01:47:10 +07:00
d37dc6e1c9 [clyp] Add support for token protected media (closes #17184) 2018-08-07 23:27:08 +07:00
a62460aa21 [imdb] Fix extension extraction (closes #17167) 2018-08-06 04:37:03 +07:00
d588d4a5a6 [.gitignore] Add .vscode 2018-08-05 22:10:01 +07:00
81cc22bab6 release 2018.08.04 2018-08-04 01:23:24 +07:00
20f96f64bd [ChangeLog] Actualize
[ci skip]
2018-08-04 01:21:23 +07:00
af322eb830 [funk:channel] Improve byChannelAlias extraction (closes #17142) 2018-08-04 00:26:58 +07:00
cb1c3a3c07 [twitch] Update cliend id and modernize (closes #17126) 2018-08-03 22:44:31 +07:00
48afc6ca3e [twitch] Fix authentication (closes #17024) 2018-08-03 22:43:23 +07:00
644921b372 [twitch:vod] Improve _VALID_URL (closes #17135) 2018-08-02 23:16:15 +07:00
19b9de13c4 [watchbox] Fix extraction (closes #17107) 2018-07-30 23:28:44 +07:00
6f2d82a5a0 [pbs] Fix extraction (closes #17109) 2018-07-30 23:10:40 +07:00
7ff129d3ea [theplatform] Relax _VALID_URL (closes #16181) 2018-07-30 03:15:06 +07:00
9d1b213845 [viqeo] Add extractor (closes #17066) 2018-07-30 03:05:36 +07:00
5484828418 release 2018.07.29 2018-07-29 07:02:18 +07:00
4eecef84f3 [ChangeLog] Actualize
[ci skip]
2018-07-29 06:59:39 +07:00
b2286f8fb2 [crunchyroll:playlist] Restrict _VALID_URL (closes #17069) 2018-07-29 06:56:52 +07:00
4938c8d573 [pornhub] Add support for subtitles (closes #16924) 2018-07-29 06:24:10 +07:00
1a88fc5a69 [ceskatelevize] Use https for API call (refs #16997) 2018-07-29 06:04:59 +07:00
38e87f6c2a [utils] Remove return from __init__ 2018-07-29 05:52:42 +07:00
ec240a4369 [dailymotion:playlist] fix extraction(closes #16894) 2018-07-28 20:30:44 +01:00
cd3a3ff93b [ted] Improve extraction and update tests 2018-07-28 22:09:53 +07:00
9a984265b9 [ted] Fix extraction for videos without nativeDownloads (closes #16756, closes #17085) 2018-07-28 21:26:23 +07:00
a098c99f0d [telecinco] fix extraction(closes #17080) 2018-07-28 06:55:42 +01:00
8e37a7e4cc [mitele] reduce number of requests and update tests 2018-07-28 06:55:42 +01:00
722f1a0f8f [README.md] Actualize Firefox cookie export add-on
Previous one does not work with newer Firefox versions
2018-07-28 00:18:41 +07:00
0c7b4f49eb [rai] return non http relinker URL intact(closes #17055) 2018-07-26 08:11:06 +01:00
ad1bc71a8a [vk] fix extraction for inline only videos(fixes #16923) 2018-07-26 07:25:07 +01:00
b5dec62ca6 [streamcloud] Fix extraction (closes #17054) 2018-07-25 23:07:12 +07:00
631f93ee2d [facebook] fix tahoe request for authenticated users(closes #16655) 2018-07-23 06:20:18 +01:00
d4e7065111 Credit @Kerruba for #16328 2018-07-22 21:36:58 +07:00
234a85858c Credit @tmsbrg for #15462 2018-07-22 21:35:38 +07:00
a789d1cc90 Credit @nathanrossi for #16554 2018-07-22 21:34:34 +07:00
694079dff7 Credit @mrfade for #16269 and #16271 2018-07-22 21:31:46 +07:00
d94fb1225e Credit @dnet for #16174 2018-07-22 21:29:25 +07:00
7930f91494 Credit @haasn for #16326 2018-07-22 21:27:28 +07:00
a702056fbe Credit @bastiandg for #16189 2018-07-22 21:26:12 +07:00
8fd2a7be37 [puhutv] Improve extraction (closes #16269) 2018-07-22 20:36:48 +07:00
6de82b4476 [puhutv] Add extractor (closes #16010) 2018-07-22 20:36:48 +07:00
8e66ffc3b7 release 2018.07.21 2018-07-21 21:00:18 +07:00
6f27998e75 [ChangeLog] Actualize
[ci skip]
2018-07-21 20:58:30 +07:00
3052a30d42 Improve URL extraction 2018-07-21 19:08:28 +07:00
4ecf300d13 [iwara] Improve extraction 2018-07-21 18:03:58 +07:00
af03000ad5 [utils] Introduce url_or_none 2018-07-21 18:03:58 +07:00
b96b4be461 [bbc] add support for BBC Radio Play pages(closes #17022) 2018-07-21 11:50:14 +01:00
edb0e17188 [iwara] Fix download URLs (closes #17026) 2018-07-21 17:41:33 +07:00
e9c671d5e8 [utils] Allow JSONP with empty func name (closes #17028) 2018-07-21 12:30:18 +07:00
fd62b36680 [vrtnu] Relax title extraction and extract JSON-LD (closes #17018) 2018-07-20 02:39:20 +07:00
25586c601c [theplatform] PEP 8
[ci skip]
2018-07-20 00:49:14 +07:00
ecb6b6ae2d [viu] Pass area id 2018-07-20 00:46:50 +07:00
c258570edd [viu] Pass Referer and Origin headers (closes #16992) 2018-07-20 00:01:43 +07:00
6fc09f0155 [vimeo] Add another config regex (closes #17013) 2018-07-19 23:15:27 +07:00
11330f5121 [facebook] Extract view count and update tests (closes #16942) 2018-07-19 02:26:05 +07:00
8da17f9680 [dailymotion] Improve description extraction (closes #16984) 2018-07-19 02:04:05 +07:00
c63f5fb863 [slutload] Fix and improve extraction (closes #17001) 2018-07-19 01:59:00 +07:00
38f1eb0ac3 [mediaset] fix extraction(closes #16977) 2018-07-18 18:34:04 +01:00
371dcc1dd4 [theplatform] add support for theplatform Top-level domain customization(#16977) 2018-07-18 18:34:04 +01:00
bd21ead2a2 [extractor/common] add support for DASH and MSS formats extraction in SMIL manifests 2018-07-18 18:34:04 +01:00
905eef2b06 [imgur] Allow digits in filename extension 2018-07-18 23:47:26 +07:00
79367a9820 [pornhub] Improve extraction and extract all formats (closes #12166, closes #15891, closes #16262, closes #16959) 2018-07-14 18:05:06 +07:00
40a051fa9f release 2018.07.10 2018-07-10 02:09:51 +07:00
7e8e948cf7 [ChangeLog] Actualize
[ci skip]
2018-07-10 02:08:15 +07:00
4b3ee09886 [nrktv] Add support for new season and serie URL schema 2018-07-10 00:27:24 +07:00
79fd7320e2 [nrktv] Add support for new episode URL schema (closes #16909) 2018-07-09 23:44:05 +07:00
0685d9727b [utils] Share JSON-LD regex 2018-07-09 23:43:05 +07:00
e06632e3fe [downloader/dash] Improve error handling (#16927) 2018-07-08 08:22:56 +07:00
69fcdb845b [frontendmasters] Fix issues and improve extraction (closes #3661, closes #16328) 2018-07-08 00:55:49 +07:00
6868d272e5 [frontendmasters] Add extractor 2018-07-08 00:49:15 +07:00
4742150788 [funk] Fix extraction (closes #16918) 2018-07-06 23:50:02 +07:00
4e71dfd819 [README.md] Rename OS X to macOS 2018-07-05 22:17:18 +07:00
1ed0b2f74d [watchbox] Fix extraction (closes #16904) 2018-07-05 02:22:15 +07:00
e15141adae [dplayit] Sort formats 2018-07-05 02:14:50 +07:00
94fef94d9c [dplayit] Fix extraction (closes #16901) 2018-07-05 02:14:06 +07:00
9a6628aaf9 [youtube] Improve login error handling (closes #13822) 2018-07-05 00:37:32 +07:00
689af4960e release 2018.07.04 2018-07-04 04:59:21 +07:00
d5de0f21b9 [ChangeLog] Actualize
[ci skip]
2018-07-04 04:57:17 +07:00
24d26ab380 [lynda] PEP 8 2018-07-04 04:49:03 +07:00
836ef4840f [pluralsight] Switch to graphql (closes #16889, closes #16899) 2018-07-04 04:48:40 +07:00
5621c3222e [lynda] Simplify login and improve error capturing (#16891) 2018-07-03 02:47:09 +07:00
db5debf313 [go90] add support for embed urls(closes #16873) 2018-07-01 22:41:32 +01:00
8cee692b8b [go90] detect geo restriction error and pass geo verification headers(closes #16874) 2018-07-01 22:41:32 +01:00
973b6ceebb [vlive] Fix live streams extraction 2018-07-01 21:19:17 +07:00
eca1f0d115 [extractor/common] Properly escape % in MPD templates (closes #16867) 2018-07-01 02:11:36 +07:00
2160768a21 [npo] Fix typo (closes #16872) 2018-06-30 23:39:56 +07:00
267d81962a [mediaset] Fix issues and extract all formats (closes #16568) 2018-06-30 02:19:02 +07:00
9cf648c92b [mediaset] Add support for new videos 2018-06-30 02:17:51 +07:00
5e8e2fa51f [extractor/common] Use source URL as Referer for HTML5 entries (closes #16849) 2018-06-29 01:25:05 +07:00
d4a24f4091 Prefer ffmpeg over avconv by default (closes #8622) 2018-06-29 01:09:14 +07:00
acbd0ff5df [dctptv] Restore extraction based on REST API (closes #16850) 2018-06-29 00:35:05 +07:00
7b393f9cc5 [svt] Improve extraction and add support for pages (closes #16802) 2018-06-28 04:29:11 +07:00
c3bcd206eb [porncom] Fix extraction (closes #16808) 2018-06-26 00:01:06 +07:00
1f6cc5807e release 2018.06.25 2018-06-25 02:26:02 +07:00
c306f076ec [ChangeLog] Actualize
[ci skip]
2018-06-25 02:17:14 +07:00
a0949fec08 [joj] Relax _VALID_URL (closes #16771) 2018-06-24 23:57:22 +07:00
74caf528bc [brightcove] workaround sonyliv DRM protected videos(closes #16807) 2018-06-24 12:02:32 +01:00
9fb62e35f6 [motherless:group] Fix _VALID_URL 2018-06-21 23:39:13 +07:00
b71cc71910 [motherless] Fix extraction (closes #16786) 2018-06-21 23:38:32 +07:00
a4ec45179e [itv] Sort imports 2018-06-21 23:12:40 +07:00
30374f4d40 [itv] Make SOAP request non fatal and extract metadata from a webpage (closes #16780) 2018-06-21 23:06:58 +07:00
91aa502d91 [foxnews:insider] Remove extractor (#15810)
Now covered by foxnews:article
2018-06-20 23:59:37 +07:00
f51f526b0a [foxnews] Add support for iframe embeds (closes #15810, closes #16711) 2018-06-20 23:53:37 +07:00
c9b983ff82 release 2018.06.19 2018-06-19 23:16:04 +07:00
e730508827 [ChangeLog] Actualize
[ci skip]
2018-06-19 23:12:53 +07:00
8b4b400aef [peertube] Improve generic support (closes #16733) 2018-06-19 23:00:36 +07:00
e12b4b8bcc [6play] use geo verfication headers 2018-06-19 10:35:57 +01:00
18806e3b6b [rtbf] fix extraction for python 3.2 and older 2018-06-18 19:10:43 +01:00
713afa705c [vgtv] Improve HLS formats extraction 2018-06-18 23:15:38 +07:00
721a877d2f [vgtv] Add support for www.aftonbladet.se/tv/ URLs 2018-06-18 23:08:35 +07:00
9283d4ea03 [bbccouk] Use expected_status 2018-06-18 04:54:59 +07:00
00a429bea3 [markiza] Expect 500 status code 2018-06-18 04:54:52 +07:00
d391b7e23d [extractor/common] Introduce expected_status for convenient accept of failed HTTP requests
Useful when some non-success (2xx) HTTP status codes should be considered normal. Previously this required to manually catch corresponding exceptions and read the response.
2018-06-18 04:54:08 +07:00
075a13d3e9 [compat] Introduce compat_integer_types 2018-06-18 04:52:58 +07:00
8ba84e4600 [tvnow] try all clear manifest urls(closes #15361) 2018-06-17 20:41:09 +01:00
858cf4dc29 release 2018.06.18 2018-06-18 01:34:36 +07:00
9e761fe6f5 [ChangeLog] Actualize
[ci skip]
2018-06-18 01:31:49 +07:00
ce0edda0f9 [markiza] Add extractors (closes #16750) 2018-06-18 01:17:47 +07:00
0adf213d8c [wat] try all supported adaptive urls 2018-06-17 15:56:52 +01:00
8b183bd5f8 [tf1] try all supported adaptive urls 2018-06-17 15:53:29 +01:00
1882511754 [6play] add support for rtlplay.be and extract hd usp formats 2018-06-17 12:01:14 +01:00
764cd4e6f3 [rtbf] improve extraction
- add support for audio and live streams(closes #11923)(closes #9638)
- extract HLS, DASH and all HTTP formats
- extract subtitles
- fixup specific http urls(fixes #16101)
2018-06-17 03:13:41 +01:00
734d461ca0 [expressen] Add extractor 2018-06-16 21:15:06 +07:00
81c5df4f2c [vidzi] Fix extraction (closes #16678) 2018-06-16 05:08:44 +07:00
87f89dacdd [pbs] Improve extraction (closes #16623, closes #16684) 2018-06-16 02:55:20 +07:00
9b0b627534 [downloader/rtmp] Fix downloading in verbose mode (closes #16736) 2018-06-15 02:59:15 +07:00
61cb66830f [bilibili] Restrict cid regex (closes #16638, closes #16734) 2018-06-14 22:40:30 +07:00
c797db4a2f release 2018.06.14 2018-06-14 01:24:53 +07:00
03eef0f032 [ChangeLog] Actualize
[ci skip]
2018-06-14 01:22:42 +07:00
aa56061627 [discoverynetworks] Add support for disco-api videos(closes #16724) 2018-06-13 16:46:59 +01:00
18d66f0410 [dailymotion] use compat_struct_pack 2018-06-13 15:12:42 +01:00
f15f7a674b [dailymotion] add support for password protected videos(closes #9789) 2018-06-13 14:51:19 +01:00
9aca7fe6a3 [abc:iview] Extract more series metadata 2018-06-12 20:25:50 +07:00
e0671819e7 [abc] fix ABC IView extraction and add support for livestreams(closes #16704)(closes #12354) 2018-06-12 13:07:57 +01:00
5d6c81b63f [downloader/http] Fix resume when writing ot stdout (closes #16699) 2018-06-12 03:12:29 +07:00
dc53c78634 [crackle] Add support for sonycrackle.com (closes #16698) 2018-06-12 02:06:30 +07:00
7dc9c60b4b [tvnet] Fix _VALID_URL 2018-06-12 02:05:58 +07:00
e51752754d [tvnet] Improve video id extraction 2018-06-12 01:50:43 +07:00
0645be49cb [inc] PEP 8 2018-06-12 01:41:23 +07:00
a572ae6114 [tvnet] Improve and fix issues (closes #15462) 2018-06-12 01:37:34 +07:00
b2df66aeca [tvnet] Add extractor 2018-06-12 01:37:29 +07:00
93cffb1444 [nrk] Update API hosts and try all previously known ones (closes #16690) 2018-06-11 03:08:36 +07:00
d253df2f65 [wimp] Fix Youtube embeds extraction 2018-06-11 02:40:17 +07:00
e8c6afc168 release 2018.06.11 2018-06-11 01:57:30 +07:00
cc37cc3f99 [ChangeLog] Actualize
[ci skip]
2018-06-11 01:55:16 +07:00
9d581efe05 [npo] Extend _VALID_URL (closes #16682) 2018-06-10 00:26:16 +07:00
ff2e486221 [inc] Add support for another embed schema (closes #16666) 2018-06-09 02:53:04 +07:00
6ae36035d9 [tv4] fix format extraction(closes #16650) 2018-06-06 00:41:08 +01:00
9afd74d705 [nexx] extract free cdn http formats 2018-06-05 01:02:46 +01:00
2e6975306a [nexx] Update tests 2018-06-05 02:59:25 +07:00
06ea7bdd99 [nexx] Add support for free cdn (closes #16538) 2018-06-05 02:55:54 +07:00
d7be705308 [pbs] Add another cove id pattern (closes #15373) 2018-06-05 00:17:26 +07:00
2e190c2ad9 [rbmaradio] Add support for 192k format (closes #16631) 2018-06-04 23:51:25 +07:00
94418c8eb3 release 2018.06.04 2018-06-04 02:41:53 +07:00
f7560859a3 [devscripts/update-copyright] Update copyright year 2018-06-04 02:33:54 +07:00
c6c478f40d [ChangeLog] Actualize
[ci skip]
2018-06-04 02:16:33 +07:00
c3023e9f2e [camtube] Add extractor 2018-06-03 17:09:20 +07:00
77053237c5 [twitter:card] Generalize base API URL 2018-06-03 15:58:12 +07:00
b6b2ccb72f [twitter:card] Extract guest token (closes #16609) 2018-06-03 15:57:45 +07:00
0a10f50e2f [chaturbate] Use geo verification headers 2018-06-03 04:30:33 +07:00
6d155707e6 [bbc] Add support for bbcthree (closes #16612) 2018-06-03 04:07:59 +07:00
eb6793ba97 [youtube] Update tests 2018-06-03 02:23:45 +07:00
7e72694b5e [youtube] Move metadata extraction after video availability check 2018-06-03 02:08:38 +07:00
936784b272 [youtube] Extract track and artist 2018-06-03 02:05:14 +07:00
003fe73ccf [safari] Add support for new URL schema (closes #16614) 2018-06-03 00:53:11 +07:00
1ea559c445 [adn] fix extraction 2018-06-02 18:14:22 +01:00
19e42ead9b release 2018.06.02 2018-06-02 01:51:31 +07:00
73c938e460 [ChangeLog] Actualize
[ci skip]
2018-06-02 01:49:48 +07:00
9b89daefa6 [facebook] Improve extraction (closes #16554) 2018-06-02 01:42:05 +07:00
9d082e7cb8 [facebook] Add support for tahoe player videos (closes #15441)
Specific videos appear to use a newer/different player, this requires a
second request for the video data as the initial request is missing the
specified data.

Additionally these videos have different page content for the uploader
value, which is stored in the `<meta property="og:title"...>` element of
the initial request.
2018-06-02 01:32:53 +07:00
f20f636596 [cbc] Improve extraction (closes #16583, closes #16593) 2018-06-02 00:35:07 +07:00
b995043ab8 Ignore venv directory 2018-06-02 00:18:57 +07:00
85750f8972 [openload] Improve ext extraction 2018-06-02 00:16:22 +07:00
926d97fc6b [9c9media] PEP 8 2018-06-01 05:17:49 +07:00
2593725a9b [twitter:card] Add support for another endpoint (closes #16586) 2018-06-01 05:16:00 +07:00
0bfdcc1495 [openload] Add support for oload.win and oload.download 2018-05-31 22:01:44 +07:00
c3f75e2454 [audimedia] fix extraction(closes #15309) 2018-05-31 12:39:45 +01:00
3a8e3730c1 [francetv] add support for sport.francetvinfo.fr(closes #15645) 2018-05-31 11:40:37 +01:00
acca2ac7f3 [mlb] improve extraction(closes #16587) 2018-05-31 02:50:14 +01:00
128b58ad13 [nhl] remove old extractors 2018-05-31 02:49:35 +01:00
4fd1437d9d [rbmaradio] check formats availability(closes #16585) 2018-05-30 17:08:32 +01:00
e425710554 release 2018.05.30 2018-05-30 21:54:30 +07:00
bc3143ac5e [ChangeLog] Actualize
[ci skip]
2018-05-30 21:52:33 +07:00
e0d42dd4b2 [teamcoco] Fix extraction for full episodes(closes #16573) 2018-05-30 13:21:07 +01:00
a07879d6b2 [spiegel] fix info extraction(#16538) 2018-05-28 00:10:46 +01:00
cfd7f2a636 [apa] Add extractor (closes #15041, closes #15672) 2018-05-27 18:24:54 +07:00
9c65c4a6cd [bellmedia] add support for bnnbloomberg.ca(#16560) 2018-05-27 12:11:53 +01:00
c9e12a618c [9c9media] extract mpd formats and subtitles 2018-05-27 12:10:12 +01:00
8882840ec5 [cammodels] Use geo verification headers 2018-05-26 22:22:58 +07:00
2ce35d9f43 [cammodels] Add another error pattern 2018-05-26 22:22:58 +07:00
f16f48779c [downloader/rtmp] Generalize download messages and report time elapsed on finish 2018-05-26 22:22:58 +07:00
ddd8486a44 [downloader/rtmp] Gracefully handle live streams interrupted by user 2018-05-26 22:22:58 +07:00
68217024e8 remove unnecessary assignment parenthesis 2018-05-26 16:13:54 +01:00
ec2f3d2800 [ufctv] add support for authentication(closes #16542) 2018-05-26 16:13:54 +01:00
8b1da46e8f [cammodels] Improve and simplify (closes #14499) 2018-05-26 21:25:30 +07:00
2a49d01992 [cammodels] Add extractor 2018-05-26 21:25:21 +07:00
261f47306c [utils] fix style id extraction for namespaced id attribute(closes #16551) 2018-05-26 14:38:24 +01:00
c0fd20abca [soundcloud] detect format extension(closes #16549) 2018-05-26 14:38:24 +01:00
986c0b0215 [cbc] Fix playlist title extraction (closes #16502) 2018-05-26 20:05:54 +07:00
97b01144bd [tumblr] Detect and report sensitive media (closes #13829) 2018-05-26 20:00:00 +07:00
56cd31f320 [tumblr] Improve authentication (closes #15133) 2018-05-26 19:59:35 +07:00
c678192af3 [tumblr] Add support for authentication 2018-05-26 19:56:01 +07:00
0934c9d4fa release 2018.05.26 2018-05-26 13:02:21 +07:00
38e4e8ab80 [ChangeLog] Actualize
[ci skip]
2018-05-26 12:58:34 +07:00
5a16c9d9d3 [utils] keep the original TV_PARENTAL_GUIDELINES dict 2018-05-25 23:12:50 +01:00
bdbcc8eecb [dvtv] Remove dead test 2018-05-26 02:15:50 +07:00
9ef5cdb5cb [audiomack] Stringify video id (closes #15310) 2018-05-26 02:13:29 +07:00
03fad17cb6 [izlesene] Improve extraction and fix issues (closes #16407, closes #16271) 2018-05-26 01:53:17 +07:00
f4d261b765 [izlesene] Fix extraction (closes #16233) 2018-05-26 01:53:11 +07:00
aee36ca832 [indavideo] Add support for generic embeds (closes #11989) 2018-05-26 01:25:40 +07:00
2a7c6befc1 [indavideo] Fix extraction (closes #11221) 2018-05-26 01:09:44 +07:00
b39f42ee92 [indavideo] Sign download URLs 2018-05-26 00:46:05 +07:00
6bd499e8ca [peertube] Add support for generic embeds 2018-05-26 00:28:30 +07:00
f2fc63a5a8 [peertube] Add support for embed and API URLs 2018-05-26 00:15:38 +07:00
c561b75c82 [peertube] Add extractor (closes #16301, closes #16329) 2018-05-26 00:09:15 +07:00
3d2a643fdc [imgur] Fix extraction 2018-05-25 21:54:21 +08:00
e8e58c2278 [hidive] add support for authentication(closes #16534) 2018-05-24 11:53:42 +01:00
1139935db7 [nbc] add support for stream.nbcsports.com(closes #13911) 2018-05-24 02:51:47 +01:00
ca0aef42d4 [viewlift] add support for hoichoi.tv(closes #16536) 2018-05-23 23:04:12 +01:00
3bb3ff38a1 [test_utils] add tests for b836118724 2018-05-23 12:20:05 +01:00
268e132dec [go90] extract age limit and detect drm protection(#10127) 2018-05-23 12:15:21 +01:00
670dcba8c7 [viewlift] Remove rating format transformation 2018-05-23 12:13:44 +01:00
b836118724 [utils] Relax TV Parental Guidelines matching 2018-05-23 12:12:20 +01:00
57d6792024 [viewlift] fix extraction for snagfils.com(closes #15766) 2018-05-23 11:27:36 +01:00
b89ac53455 [globo] use compat_str 2018-05-21 17:46:52 +01:00
d81ffc3aa0 [globo] Add entry for netrc authentication 2018-05-21 15:39:02 +01:00
e518749300 [globo] handle login errors 2018-05-21 15:07:24 +01:00
db2058f63e [globo] improve extraction(closes #4189)
- add support for authentication
- simplify url signing
- extract DASH and MSS formats
2018-05-21 14:55:50 +01:00
5c766952dc Update leeco.py
Fixed this bug :youtube_dl.utils.ExtractorError: An extractor error has occurred. (caused by KeyError('location',));
2018-05-21 21:26:53 +08:00
504f20dd30 Remove experimental mark for some options 2018-05-19 23:53:24 +07:00
f2b1fa07ec [teamcoco] relax _VALID_URL regex and add a fallback for format extraction(fixes #16484) 2018-05-19 13:05:51 +01:00
acd620c930 [teamcoco] improve _VALID_URL regex(#16484) 2018-05-19 12:19:05 +01:00
27694fe7ad [imdb:list] fix _VALID_URL regex 2018-05-19 11:04:08 +01:00
0167f0dbfe [imdb] improve extraction(fixes #4085)(fixes #14557) 2018-05-19 10:15:11 +01:00
7550ea501a release 2018.05.18 2018-05-18 00:32:51 +07:00
58197205d3 [ChangeLog] Actualize
[ci skip]
2018-05-18 00:30:41 +07:00
361a965b5c [vimeo:likes] Relax _VALID_URL and fix single page likes extraction (closes #16475) 2018-05-17 23:21:40 +07:00
a3f86160fa [pluralsight] fix clip id extraction(fixes #16460) 2018-05-17 13:46:05 +01:00
1306f5ed72 [mychannels] add support for mychannels.com(closes #15334) 2018-05-16 19:11:48 +01:00
58a68d8fda [moniker] Remove extractor(closes #15336) 2018-05-16 18:44:33 +01:00
eea2fafcf5 [pbs] fix embed data extraction(fixes #16474) 2018-05-16 18:34:25 +01:00
6843ac5b13 add support for paramountnetwork.com and bellator.com(fixes #15418) 2018-05-16 17:49:35 +01:00
54fc90aabf [youtube] fix hd720 format position 2018-05-16 16:24:44 +01:00
997530d9d4 [dailymotion] remove fragment part from m3u8 urls(closes #8915) 2018-05-16 12:04:24 +01:00
fe3a60f040 [dreisat] improve extraction(closes #15350)
- extract all formats
- extract more format metadata extraction
- improve format sorting
- use hls native downloader
- detect geo-restriction
- bypass geo-restriction
2018-05-16 11:30:29 +01:00
7f34984e81 [dtube] Add new extractor(closes #15201) 2018-05-16 09:35:47 +01:00
1e4fe5a7cc [options] Fix typo (closes #16450) 2018-05-14 23:42:33 +07:00
c63ca0eef8 [youtube] Improve format filesize extraction (#16453) 2018-05-14 23:27:56 +07:00
84a9fef899 [youtube] Make uploader extraction non fatal (#16444) 2018-05-13 22:49:01 +07:00
4c76aa0666 [youtube] fix extraction for embed restricted live streams(fixes #16433) 2018-05-13 13:20:16 +01:00
90b633f86b [nbc] improve info extraction(fixes #16440) 2018-05-13 11:31:41 +01:00
07acdc5afc [twitch:clips] Sort formats 2018-05-12 12:08:54 +07:00
49fa7de301 [twitch:clips] Fix extraction (closes #16429) 2018-05-11 23:21:02 +07:00
dbd5c502ea [redditr] Relax _VALID_URL (closes #16426) 2018-05-10 23:17:23 +07:00
bc5e4aa57e [mixcloud] Bypass throttling for HTTP formats (#12579, #16424) 2018-05-10 22:22:26 +07:00
1344d3e169 [nickbr] Relax _VALID_URL (#13230) 2018-05-10 22:01:13 +07:00
ff8889cd4d [teamcoco] fix extraction(closes #16374) 2018-05-10 08:19:56 +01:00
9e18bb4c67 release 2018.05.09 2018-05-09 00:36:47 +07:00
44277998ad [ChangeLog] Actualize
[ci skip]
2018-05-09 00:34:39 +07:00
05108a496a [YoutubeDL] Ensure ext exists for automatic captions 2018-05-08 22:57:52 +07:00
2fbd86352e [udemy] Extract asset captions 2018-05-08 22:57:01 +07:00
0ce76801e8 [udemy] Extract stream URLs (closes #16372) 2018-05-08 22:33:35 +07:00
789b7774a7 [businessinsider] Add extractor (closes #16387, closes #16388, closes #16389) 2018-05-06 21:58:55 +07:00
660a230b2d [cloudflarestream] Add support for cloudflare streams (closes #16375) 2018-05-05 01:21:52 +07:00
a90a6b54ee [watchbox] Fix extraction (closes #16356) 2018-05-02 20:43:34 +07:00
3cc0d0b829 [discovery] extract Affiliate/Anonymous Auth Token from cookies(closes #14954) 2018-05-02 09:32:53 +01:00
ea1f5e5dbd [itv:btcc] Add extractor (closes #16139) 2018-05-02 07:21:24 +07:00
5f95927a62 Improve geo bypass mechanism
* Introduce geo bypass context
* Add ability to bypass based on IP blocks in CIDR notation
* Introduce --geo-bypass-ip-block
2018-05-02 07:20:59 +07:00
a93ce61bd5 [tunein] Use live title for live streams (closes #16347) 2018-05-02 01:29:44 +07:00
c18142da6e [itv] Improve extraction (closes #16253) 2018-05-01 22:48:08 +07:00
cc42941390 release 2018.05.01 2018-05-01 03:38:57 +07:00
cc5772c4f0 [ChangeLog] Actualize
[ci skip]
2018-05-01 03:30:23 +07:00
c21692fa94 [kaltura] Improve iframe embeds detection (closes #16337) 2018-05-01 03:09:04 +07:00
8513963468 [udemy] Extract outputs renditions (closes #16289, closes #16291, closes #16320, closes #16321, closes #16334, closes #16335) 2018-05-01 02:15:43 +07:00
67ca1a8ef7 [zattoo] Improve and simplify (closes #14676) 2018-05-01 01:50:30 +07:00
4a73354586 [zattoo] Add extractor (closes #14668) 2018-05-01 01:50:07 +07:00
796bf9de45 [yandexmusic] Convert release_year to int 2018-04-29 22:56:07 +07:00
e5eadfa82f [udemy,xiami,yandexmusic] Override _download_webpage_handle instead of _download_webpage 2018-04-29 22:54:52 +07:00
30226342ab [youtube] Correctly disable polymer on all requests
Rather than just the one that use the _download_webpage helper. The need
for this was made apparent by 0fe7783e, which refactored
_download_json in a way that completely avoids the use of
_download_webpage, thus breaking youtube.

Fixes #16323
2018-04-29 22:35:16 +07:00
01aec84880 [generic] Prefer enclosures over links in RSS feeds 2018-04-29 22:14:37 +07:00
12b0d4e0e1 [redditr] Add support for old.reddit.com URLs 2018-04-29 21:59:40 +07:00
106c8c3edb [nrktv] Update API host (closes #16324) 2018-04-29 19:04:40 +07:00
500a86a52e [downloader/fragment] Restart download if .ytdl file is corrupt (closes #16312) 2018-04-29 00:33:31 +07:00
7dd6ab4a47 [imdb] Extract all formats (closes #16249) 2018-04-28 04:51:39 +07:00
ae1c585cee [vimeo] Extract JSON LD (closes #16295) 2018-04-28 02:51:18 +07:00
e7e4a6e0f9 [extractor/common] Extract interaction statistic 2018-04-28 02:48:03 +07:00
6cc622327f [utils] Introduce merge_dicts 2018-04-28 02:47:17 +07:00
0fe7783ece [extractor/common] Add _download_json_handle 2018-04-28 01:59:15 +07:00
c84eae4f66 [funk:channel] Improve extraction (closes #16285) 2018-04-27 03:45:52 +07:00
d3711b0050 [devscripts/gh-pages/generate-download.py] Use program checksum from versions.json 2018-04-25 02:14:27 +07:00
b5802d69f5 release 2018.04.25 2018-04-25 01:12:40 +07:00
e028d4f506 [ChangeLog] Actualize
[ci skip]
2018-04-25 01:07:37 +07:00
ecb24f7c08 Credit @f2face for #16115 2018-04-25 01:07:32 +07:00
95284bc281 Credit @TingPing for picarto (#15551) 2018-04-25 01:07:27 +07:00
5d0fe6d23e Credit @Zopieux for #16250 2018-04-25 01:07:23 +07:00
76030543cd [openload] Recognize IPv6 stream URLs (closes #16137) 2018-04-25 00:49:30 +07:00
0ff51adae6 [twitch] Extract is_live according to status (closes #16259) 2018-04-24 23:55:06 +07:00
1cc47c6674 [utils] Fix match_str for boolean meta fields 2018-04-24 23:54:49 +07:00
99036a1298 [pornflip] Relax _VALID_URL (closes #16258) 2018-04-23 04:03:11 +07:00
171625469a [etonline] Remove extractor (closes #16256)
Covered by generic extractor
2018-04-23 03:17:34 +07:00
af751350e8 [Makefile] Add support for pandoc 2 and disable smart extension (closes #16251)
smart extension rewrites straight quotes as curly quotes, -- as en-dashes and so on that is unwanted behavior.
2018-04-23 02:50:11 +07:00
2441c1aab1 [breakcom] Fix extraction (closes #16254) 2018-04-23 00:16:52 +07:00
70d35d166c [youtube] Add ability to authenticate with cookies 2018-04-22 06:08:05 +07:00
3853309fe2 [youtube:feed] Implement lazy playlist extraction (closes #10184) 2018-04-22 06:07:32 +07:00
6cdaaf7031 [svt] Improve (closes #15809) 2018-04-22 05:34:03 +07:00
488ff2dd3a [svt] Add support for TV channel live streams (Closes #15279) 2018-04-22 05:33:40 +07:00
353f0bde78 [cbssports] PEP 8 2018-04-22 04:57:22 +07:00
040c6296bb [ccma] Fix video extraction (closes #15931) 2018-04-22 04:55:35 +07:00
a693386df1 [rentv] Improve extraction (closes #15227) 2018-04-21 23:22:30 +07:00
4b8588fe02 [rentv] Fix extraction 2018-04-21 23:22:25 +07:00
d65a48a0ef [nick] Add support for nickjr.nl (closes #16230) 2018-04-20 23:12:13 +07:00
c194200277 [mofosex] Fix test 2018-04-19 22:38:31 +07:00
d317973284 [extremetube] Fix metadata extraction 2018-04-19 22:36:33 +07:00
1792bc3a06 [keezmovies] Add support for generic embeds (closes #16134) 2018-04-19 22:25:51 +07:00
5a19d231ca [YoutubeDL] Fix typo in media extension compatibility checker 2018-04-19 22:21:50 +07:00
d86c5167ae [nexx] extract new azure urls(closes #16223) 2018-04-19 15:48:03 +01:00
b004d9bbf1 [cbssports] fix extraction(fixes #16217) 2018-04-19 15:08:17 +01:00
9b3036bd2e [instagram:user] Fix extraction (closes #16119) 2018-04-18 10:12:24 +07:00
e30991f920 [kaltura] Improve embeds detection (closes #16201) 2018-04-18 01:26:15 +07:00
518d5ba519 Fix some tests 2018-04-18 00:10:02 +07:00
238d42cf5d [instagram:user] Fix extraction (closes #16119) 2018-04-17 22:37:50 +07:00
522d6b5c96 [cbs] skip DRM asset types(fixes #16104) 2018-04-16 07:48:51 +01:00
3c92fd1cd5 release 2018.04.16 2018-04-16 01:09:18 +07:00
bdf7ba6f3a Set chmod 644 for all extractors 2018-04-16 01:07:21 +07:00
0e6ccb3905 [ChangeLog] Actualize
[ci skip]
2018-04-16 00:56:05 +07:00
c07cb68e79 [smotri:broadcast] Fix extraction (closes #16180) 2018-04-16 00:54:21 +07:00
a42839e548 [picarto] Improve extraction (closes #6205, closes #12514, closes #15276, closes #15551) 2018-04-16 00:34:47 +07:00
d6166a7602 [picarto] Add extractor 2018-04-16 00:32:15 +07:00
8e41c9ad01 [vine:user] Improve extraction (closes #16190) 2018-04-15 22:46:43 +07:00
9b5aead6aa [vine:user] Fix extraction (closes #15514) 2018-04-15 22:46:30 +07:00
68ddba20ae [instagram:user] Remove User-Agent from signature (closes #16119) 2018-04-13 22:28:33 +07:00
92ded33a05 [pornhub] Relax _VALID_URLs (closes #16165) 2018-04-12 04:53:45 +07:00
64f03e5b4c [cbc:watch] Re-acquire device token when expired (closes #16160) 2018-04-11 23:30:19 +07:00
d783aee56a [fxnetworks] Add support for https theplatform URLs (closes #16125) 2018-04-11 20:11:24 +07:00
315ab3d500 [instagram:user] Simplify signing (#16119) 2018-04-11 01:51:57 +07:00
dd9aea8cbd [instagram:user] Add request signing (closes #16119) 2018-04-11 01:25:41 +07:00
fce7962691 [twitch] Add support for mobile URLs (closes #16146) 2018-04-10 23:07:37 +07:00
f7f9757efc release 2018.04.09 2018-04-09 01:19:27 +07:00
880ed89d49 [ChangeLog] Actualize
[ci skip]
2018-04-09 01:14:47 +07:00
94c3442e6a [YoutubeDL] Do not save/restore console title while simulate (closes #16103) 2018-04-09 01:04:22 +07:00
069937151e [generic] Add support for tube8 embeds 2018-04-09 00:37:15 +07:00
d3431dcb90 [generic] Restrict share-videos.se embeds regex to filter bogus URLs (#16115) 2018-04-09 00:25:44 +07:00
1fc37ca3f1 [generic] Add support for share-videos.se embeds (closes #16089) 2018-04-09 00:19:23 +07:00
d04ca97616 [odnoklassniki] Improve _VALID_URL readability 2018-04-08 22:21:21 +07:00
608c738c7d [odnoklassniki] Extend _VALID_URL (closes #16081) 2018-04-08 22:13:00 +07:00
66b686727b [extractor/common] Relax JSON-LD context check (closes #16006) 2018-04-08 03:09:42 +07:00
717ea4e14e [steam] Bypass mature content check (closes #16113) 2018-04-08 00:29:43 +07:00
cae5d9705c [acast] Extract more metadata 2018-04-08 00:21:55 +07:00
1c9b1a4494 [acast] Fix extraction (closes #16118) 2018-04-08 00:08:45 +07:00
ff826177cc [instagram:user] Fix extraction (closes #16119) 2018-04-07 23:58:57 +07:00
9d15be3a5b [drtuber] Fix title extraction (closes #16107) 2018-04-07 21:39:21 +07:00
e2750e1437 [liveleak] Extend _VALID_URL (closes #16117) 2018-04-07 20:55:01 +07:00
e944737c59 [openload] Add support for oload.xyz 2018-04-06 23:40:15 +07:00
fdfb32a0dd [openload] Relax stream URL regex 2018-04-06 00:15:22 +07:00
235d828b7b [openload] Fix extraction (closes #16099) 2018-04-05 23:49:15 +07:00
1236ac6b0b [svtplay] Share svtplay regex 2018-04-05 00:29:13 +07:00
df146eb282 [svtplay:series] Add support for season URLs 2018-04-05 00:29:08 +07:00
b71bb3ba8b [svtplay:series] Improve extraction (closes #16059) 2018-04-05 00:29:02 +07:00
fd97fa7bfc [svtplay:series] Add extractor
Related to #11130
2018-04-05 00:28:58 +07:00
e8dfecb384 release 2018.04.03 2018-04-03 00:26:11 +07:00
10f9caec04 [ChangeLog] Actualize
[ci skip]
2018-04-03 00:23:03 +07:00
ea6679fbeb [tvnow] Fix issues, simplify and improve (closes #15837) 2018-04-03 00:08:22 +07:00
3acae1e031 [tvnow] Add support for shows 2018-04-03 00:06:47 +07:00
8bd1df3c31 [dramafever] Fix authentication (closes #16067) 2018-04-02 22:19:42 +07:00
86693c4930 [afreecatv] Use partial view only when necessary (closes #14450) 2018-04-02 00:00:45 +07:00
d563fb32ba [afreecatv] Remove debug output 2018-04-01 23:07:54 +07:00
e51762be19 [afreecatv] Add support for authentication (#14450) 2018-04-01 22:47:39 +07:00
03fcde10ce [nationalgeographic] Add support for new URL schema (closes #16001) 2018-04-01 21:22:51 +07:00
95a1322bc1 [bilibili] Remove debug from player params regexes 2018-04-01 02:06:58 +07:00
0669f8fd8f [xvideos] Fix thumbnail extraction (closes #15978) 2018-03-31 23:46:08 +07:00
0b4bbcdcb6 [medialaan] Fix vod id 2018-03-31 22:14:49 +07:00
3e78d23b57 [openload] Add support for oload.site 2018-03-30 23:25:43 +07:00
190f6c936b [naver] Fix extraction (closes #16029) 2018-03-29 23:49:09 +07:00
02f6ccbce3 [dramafever] Partially switch to API v5 (closes #16026) 2018-03-29 23:06:13 +07:00
5d60b99717 [options] Mention comments support in --batch-file 2018-03-27 22:25:29 +07:00
9e6a418015 [abc:iview] Unescape title and series meta fields 2018-03-27 22:08:40 +07:00
99c3091850 [videa] Extend _VALID_URL 2018-03-27 22:02:04 +07:00
bbd9d8c170 release 2018.03.26.1 2018-03-26 22:32:03 +07:00
c3cfc71a0c [ChangeLog] Actualize
[ci skip]
2018-03-26 22:30:11 +07:00
671e241bfb release 2018.03.26 2018-03-26 05:03:47 +07:00
29d9594561 [ChangeLog] Actualize
[ci skip]
2018-03-26 22:11:01 +07:00
f0298f653e [downloader/external] Simplify finished progress hook reporting and add elapsed time (closes #10876) 2018-03-24 16:35:21 +07:00
2ea212628e [downloader/common] Improve progress reporting when no total bytes available 2018-03-24 16:35:15 +07:00
80aa246094 [downloader/external] Fix download finalization when writing file to stdout (closes #10809)
An OSError or IOError generally indicates something a little more
wrong than a "simple" UnavailableVideoError, so print the actual
traceback that leads to the exception. Otherwise meaningful postmortem
debugging a bug report is essentially infeasible.
2018-03-24 16:34:55 +07:00
0ff2c1ecb6 [downloader/fragment] Fix download finalization when writing file to stdout (closes #15799) 2018-03-24 16:04:23 +07:00
16132cff72 [vrv] Fix extraction on python2 (closes #15928) 2018-03-24 14:57:34 +07:00
86e1958944 [afreecatv] Update referrer (closes #15947) 2018-03-24 14:21:08 +07:00
b015cb1af3 [24video] Add support for 24video.sexy (closes #15973) 2018-03-24 14:11:27 +07:00
7d34016fb0 [crackle] Bypass geo restriction 2018-03-24 01:49:50 +07:00
b9f5a41207 [crackle] Fix extraction (closes #15969) 2018-03-23 23:53:18 +07:00
8b7340a45e [lenta] Add extractor (closes #15953) 2018-03-22 23:07:31 +07:00
1d4a0520ba Merge pull request #15939 from sudovijay/patch-11
[Youku] Update ccode
2018-03-22 14:42:35 +08:00
cba5d1b6b3 [instagram:user] Add pagination (closes #15934) 2018-03-21 23:43:03 +07:00
328ddf56a1 [Youku] Update ccode 2018-03-21 12:13:31 +05:30
3395958d2b libsyn: adapt to new page structure and replace testcase 2018-03-20 23:07:11 +01:00
a66d1d079a release 2018.03.20 2018-03-20 01:55:48 +07:00
c651de39d5 [ChangeLog] Actualize
[ci skip]
2018-03-20 01:54:35 +07:00
d9e2240f7c [7plus] Extract series metadata (closes #15862, closes #15906) 2018-03-20 01:40:53 +07:00
832f9d5258 [9now] Bypass geo restriction (closes #15920) 2018-03-20 01:08:03 +07:00
21dedcb580 [cbs] Skip unavailable assets (closes #13490, closes #13506, closes #15776) 2018-03-20 00:27:39 +07:00
6780154e6b [extractor/common] Improve thumbnail extraction for HTML5 entries 2018-03-19 23:43:53 +07:00
38f59e2793 [canalc2] Add support for HTML5 videos (closes #15916, closes #15919) 2018-03-19 23:40:19 +07:00
9a054fcbba [ceskatelevize] Add support for iframe embeds (closes #15918) 2018-03-19 23:29:53 +07:00
6e3f23d912 [prosiebensat1] Add support for galileo.tv (closes #15894) 2018-03-19 04:14:33 +07:00
47a5cb7734 Generalize XML manifest processing code and improve XSPF parsing (closes #15794) 2018-03-18 02:52:17 +07:00
e0d198c18d [extractor/common] Add _download_xml_handle 2018-03-18 02:52:01 +07:00
96b8b9abae [extractor/generic] Support relative URIs in _parse_xspf
<location> can have relative URIs, not just absolute.
2018-03-18 02:48:44 +07:00
178ee88319 [generic] Add support for xfileshare embeds (closes #15879) 2018-03-17 23:57:07 +07:00
d123960857 [bilibili] Switch to v2 playurl API 2018-03-16 03:18:53 +07:00
3526c3043b [bilibili] Fix and improve extraction (closes #15048, closes #15430, closes #15622, closes #15863) 2018-03-16 00:19:17 +07:00
8e70c1bfac [heise] Improve extraction (closes #15496, closes #15784, closes #15026) 2018-03-15 23:09:24 +07:00
27b1c73f14 [instagram] fix user videos extraction(fixes #15858) 2018-03-15 14:33:36 +01:00
46c6742d4f release 2018.03.14 2018-03-14 22:49:22 +07:00
c95dfb0509 [ChangeLog] Actualize
[ci skip]
2018-03-14 22:45:05 +07:00
b8c6badc96 [soundcloud] Update client id (closes #15866) 2018-03-14 22:38:55 +07:00
b848a4ca1a [tennistv] Remove duplicate key in dictionary 2018-03-14 11:48:26 +01:00
e6e68069f6 [tennistv] Correctly encode POST parameters
In python 3.x, the POST parameters must be bytes, not str.
2018-03-14 11:23:10 +01:00
f3672ac522 [line] lint (remove space on empty line) 2018-03-14 09:55:46 +01:00
f226880c6d [tennistv] Add support for tennistv.com 2018-03-14 09:55:21 +01:00
08250b69c2 [line] Add support for tv.line.me (closes #9427) 2018-03-11 22:25:30 +08:00
d116918993 [raywenderlich] PEP 8
[ci skip]
2018-03-10 23:15:11 +07:00
7399ca1f80 [pornhub] PEP 8
[ci skip]
2018-03-10 23:14:37 +07:00
b4a190fe2a [xnxx] Fix extraction (closes #15817) 2018-03-10 23:13:14 +07:00
cc7f6c720e [njpwworld] Fix authentication (closes #15815) 2018-03-10 22:35:47 +07:00
3a0ceb32e2 release 2018.03.10 2018-03-10 04:45:57 +07:00
7dee417127 [ChangeLog] Actualize
[ci skip]
2018-03-10 04:44:46 +07:00
5b1d158834 [raywenderlich] Extract videos in order 2018-03-10 04:31:51 +07:00
a7298f3e99 [pornhub] Don't override session cookies (closes #15697) 2018-03-09 23:57:32 +07:00
5d49d879cc [raywenderlich] Add extractor (#15251) 2018-03-09 23:27:44 +07:00
b5434b5c31 [nexx] Fix typo 2018-03-08 03:25:04 +07:00
690404a6f8 [funk] Fix extraction and rework extractors (closes #15792) 2018-03-08 03:17:46 +07:00
d91dd0ce19 [nexx] Restore reverse engineered approach 2018-03-08 03:16:21 +07:00
6202f08e1b [heise] Add support for kaltura embeds (closes #14961) 2018-03-06 23:10:01 +07:00
574e9db2b0 [tvnow] Extract series metadata (closes #15774) 2018-03-06 23:06:00 +07:00
2e25f80d5d [ruutu] Continue formats extraction on NOT-USED URLs 2018-03-06 02:01:04 +07:00
64f34528df [vrtnu] Use redirect URL for building video JSON URL (closes #15767, closes #15769) 2018-03-05 22:57:19 +07:00
26ad6bcdfc [vimeo] Modernize login code and improve error messaging 2018-03-05 22:45:47 +07:00
81dc74966a [archiveorg] Fix extraction (closes #15770, closes #15772) 2018-03-05 22:30:32 +07:00
d53b6764d0 [hidive] Remove proxy from params 2018-03-04 23:23:30 +07:00
62f49dd3b9 [hidive] Add extractor (closes #15494) 2018-03-04 17:46:36 +07:00
f9f10268c1 [afreecatv] Detect deleted videos 2018-03-04 03:13:45 +07:00
f241a97312 [afreecatv] Fix extraction (closes #15755) 2018-03-04 03:01:58 +07:00
86c8cfc555 [vice] Fix extraction and rework extractors (closes #11101, closes #13019, closes #13622, closes #13778) 2018-03-03 23:08:43 +07:00
c01db237b5 [vidzi] Add support for vidzi.si (closes #15751) 2018-03-03 20:16:55 +07:00
0093c77032 [downloader/hls] Skip uplynk ad fragments (closes #15748) 2018-03-03 20:00:25 +07:00
5616caf852 [npo] Fix typo 2018-03-03 01:47:09 +07:00
05a7ffb126 release 2018.03.03 2018-03-03 01:37:01 +07:00
28f21c9501 [ChangeLog] Actualize
[ci skip]
2018-03-03 01:32:21 +07:00
4c780fbd0a [yapfiles] Add extractor (closes #15726, refs #11085) 2018-03-03 01:24:36 +07:00
7773a92800 [spankbang] Fix formats extraction (closes #15727) 2018-03-02 23:39:20 +07:00
b871d7e954 [utils] Add parse_resolution 2018-03-02 23:39:04 +07:00
44dc11db61 [adn] fix format extraction(#15716) 2018-02-28 19:41:30 +01:00
949faa15e8 [toggle] Extract DASH and ISM formats (closes #15721) 2018-02-28 22:55:09 +07:00
0c3e5f4921 Revert "Respect --prefer-insecure while updating (closes #15497)"
This reverts commit 7d2b4aa047.
2018-02-27 22:30:08 +07:00
266fbd6b73 [nickelodeon] Add support for nickelodeon.com.tr (closes #15706) 2018-02-26 22:10:44 +07:00
d1b6187012 [npo] Validate and filter format URLs (closes #15709) 2018-02-26 21:50:51 +07:00
6ab35f5e16 release 2018.02.26 2018-02-26 04:23:38 +07:00
32ae31847f [ChangeLog] Actualize 2018-02-26 04:19:04 +07:00
abe8766c35 [udemy] Use custom User-Agent (closes #15571) 2018-02-26 04:12:53 +07:00
eaa3172672 release 2018.02.25 2018-02-25 20:38:10 +07:00
797c9284d6 [ChangeLog] Actualize 2018-02-25 20:35:52 +07:00
8c73ef37b6 [vidlii] Add extractor (closes #14472, closes #14512, closes #14779) 2018-02-25 20:28:40 +07:00
b5cbe3d652 [postprocessor/embedthumbnail] Skip embedding when there aren't any thumbnails 2018-02-25 19:33:13 +07:00
ece12e6348 [streamango] Skip dead test 2018-02-25 18:36:25 +07:00
ff274e3c16 [streamango] Capture and output error messages 2018-02-25 18:34:52 +07:00
c106237d56 [streamango] Fix formats extraction, improve and simplify (closes #14256) 2018-02-25 18:27:23 +07:00
6e72ea4775 [streamango] Fix extraction (closes #14160) 2018-02-25 18:26:48 +07:00
d6a0350253 [ard] Remove dead tests 2018-02-25 17:41:12 +07:00
ad29ef043e [ard] Add alive tests 2018-02-25 17:38:07 +07:00
f01df14c4f [telequebec:emission] Extend _VALID_URL 2018-02-25 17:05:39 +07:00
9306b0c8d9 [telequebec] Add support for emissions and refactor (closes #14649, closes #14655) 2018-02-25 16:54:12 +07:00
f4b7427279 [extractor/common] Improve jwplayer subtitles extraction (closes #15695) 2018-02-25 00:59:29 +07:00
300148b48a [telequebec:live] Add extractor (closes #15688) 2018-02-24 06:17:29 +07:00
2d17c63140 [abcnews] Update tests 2018-02-24 05:17:21 +07:00
f2908d072e [mailru:music] Add extractor (closes #15618) 2018-02-24 04:52:55 +07:00
5e7841932c [aenetworks] switch to akamai hls formats(closes #15612) 2018-02-23 08:23:55 +01:00
870f3bfc63 [ytsearch] Fix flat title extraction (closes #11260, closes #15681) 2018-02-23 03:43:42 +07:00
3d977fe4d2 release 2018.02.22 2018-02-22 23:50:35 +07:00
f075838728 [ChangeLog] Actualize 2018-02-22 23:48:58 +07:00
2acc11d771 [vidio] Fix HLS URL extraction (closes #15675) 2018-02-22 22:50:39 +07:00
0704306e1d [nexx] Add support for arc.nexx.cloud URLs 2018-02-22 22:31:28 +07:00
9dc7ea320d [nexx] Don't capture domain id and add support for domainless shortcuts 2018-02-22 22:27:19 +07:00
e231afb14f [nexx] switch to ark api(closes #15652) 2018-02-22 10:41:47 +01:00
12acb9a6fb [zdf] Update tests 2018-02-21 21:57:34 +07:00
18ebd1a843 [redtube] Fix duration extraction and update test 2018-02-21 21:55:28 +07:00
8315ee6c4c [reddit] Update test 2018-02-21 04:12:56 +07:00
b9d1a79426 [9gag] Update test 2018-02-20 22:28:54 +07:00
09f934b009 [vk] Update test 2018-02-20 22:21:10 +07:00
73af6e22fd [vimeo] Update test 2018-02-20 22:20:15 +07:00
77e499f95e [xhamster] Update test 2018-02-20 22:18:50 +07:00
befa4708fd [utils] Fixup some common URL's typos in sanitize_url (closes #15649) 2018-02-19 22:50:23 +07:00
90830004c8 [sonyliv] Respect referrer (closes #15648) 2018-02-19 22:29:08 +07:00
18d7aa6efa [brightcove:new] Use referrer for formats' HTTP headers 2018-02-19 22:28:27 +07:00
b12cf31bb1 [cbc] add new extractor for olympics.cbc.ca(closes #15535) 2018-02-19 09:02:23 +01:00
7d2b4aa047 Respect --prefer-insecure while updating (closes #15497) 2018-02-18 16:43:54 +07:00
38662dfec7 [fusion] Add support for fusion.tv 2018-02-17 20:54:52 +07:00
ee706f1009 [npo] Improve quality metadata extraction 2018-02-17 20:32:34 +07:00
c4e7496421 [npo] Relax _VALID_URL (closes #14987, closes #14994) 2018-02-17 20:32:26 +07:00
b8adcec4ea [npo] Capture and output error message 2018-02-17 20:32:20 +07:00
073cca3df8 [downloader/common] Add whitespace 2018-02-17 19:11:46 +07:00
f66df20ccd [pornhub] Add support for channels (closes #15613) 2018-02-17 01:17:06 +07:00
ea69624992 [youtube] Handle shared URLs with generic extractor (closes #14303) 2018-02-15 22:33:11 +07:00
49702e3669 [francetv] Fix typo 2018-02-12 00:25:42 +07:00
59b5e7b280 release 2018.02.11 2018-02-11 22:31:52 +07:00
b9683400cf [ChangeLog] Actualize 2018-02-11 22:16:42 +07:00
760f81212f [francetv] Add support for live streams (closes #13689) 2018-02-11 21:58:04 +07:00
79080573b5 [francetv] Add support for zouzous.fr and ludo.fr (closes #10454, closes #13087, closes #13103, closes #15012) 2018-02-11 21:40:38 +07:00
99892e9908 [francetv] Separate main extractor and rework others to delegate to it 2018-02-11 21:14:05 +07:00
8faa338ff3 [francetv] Improve manifest URL signing (closes #15536) 2018-02-11 20:06:37 +07:00
818df33fda [francetv] Sign m3u8 manifest URLs (closes #15565)
they seem to be required now, else we get 403 errors.
2018-02-11 20:06:37 +07:00
a072a12e24 [veoh] Add support for embed URLs 2018-02-11 06:51:10 +07:00
e67734dda9 [dvtv] Skip download on failing test 2018-02-11 04:56:03 +07:00
9e36fedd00 [afreecatv] Fix extraction (closes #15556) 2018-02-10 22:36:26 +07:00
d2b200eef9 [periscope] Use accessVideoPublic endpoint (closes #15554) 2018-02-10 17:23:37 +07:00
a03a3c80fe [YoutubeDL] Add support for filesize_approx in format selector (closes #15550) 2018-02-10 16:42:45 +07:00
9d5871fdff [discovery] Fix auth request (closes #15542) 2018-02-10 00:55:11 +07:00
ff873b5777 [6play] Extract subtitles (closes #15541) 2018-02-10 00:04:08 +07:00
042968ff31 Credit @mweinelt for #15124 2018-02-09 23:10:36 +07:00
430f2ca544 Credit @iamleot for internazionale (#14973) 2018-02-09 23:10:30 +07:00
cbfbf07cdc Credit @che0 for seznamzpravy (#14616) and dvtv (#15442) 2018-02-09 23:10:24 +07:00
9e167e1ee3 [newgrounds] Fix metadata extraction (closes #15531) 2018-02-09 21:17:02 +07:00
5828489072 [nbc] add support for NBC Olympics Streams(closes #10295) 2018-02-09 02:05:28 +01:00
8c5fafe29f [dvtv] Simplify (closes #15442) 2018-02-09 02:19:29 +07:00
6f4ad0db34 [dvtv] Fix live streams extraction
(also fixed test broken by changed og tags on the site)
2018-02-09 02:19:05 +07:00
c2b3bd0451 release 2018.02.08 2018-02-08 23:39:40 +07:00
728cee5385 [ChangeLog] Actualize 2018-02-08 23:37:49 +07:00
246a75b4ff [extractors] Import for myvi:embed 2018-02-08 23:05:13 +07:00
4fac463d70 [pokemon] PEP 8 2018-02-08 23:04:20 +07:00
382b8182ce [gameinformer] PEP 8 2018-02-08 23:03:57 +07:00
ce53320b11 [myvi] Extend _VALID_URL 2018-02-08 23:02:05 +07:00
51b0557d1e [myvi:embed] Add extractor (closes #15521) 2018-02-08 22:57:33 +07:00
5a5860825d [prosiebensat1] Extend _VALID_URL (closes #15520) 2018-02-08 22:28:56 +07:00
237d07f114 [pokemon] Relax _VALID_URL and extend title extraction (closes #15518) 2018-02-08 03:58:35 +07:00
9f4ec3de25 [gameinformer] Use geo verification headers 2018-02-07 21:52:15 +07:00
96a0bbdd0d [la7] Fix extraction (closes #15501) 2018-02-07 21:48:16 +07:00
c8064d4fab [gameinformer] Fix brightcove id extraction 2018-02-07 21:40:48 +07:00
fde677fed4 [afreecatv] Pass referrer to video info request (closes #15507) 2018-02-06 22:21:10 +07:00
0e0508c8a2 [telebruxelles] Relax _VALID_URL and add support for live streams 2018-02-05 23:56:00 +07:00
bcf150e435 [telebruxelles] Fix extraction (closes #15504) 2018-02-05 23:56:00 +07:00
240f26229d [extractor/common] Respect secure schemes in _extract_wowza_formats 2018-02-05 23:56:00 +07:00
b9b150def7 release 2018.02.04 2018-02-04 08:03:00 +07:00
d20225f33b [ChangeLog] Actualize 2018-02-04 08:01:09 +07:00
5399ab3f0c [brightcove] Pass embed page URL as referrer (closes #15486) 2018-02-04 07:55:04 +07:00
b91a7a4e5e [downloader/http] Randomize HTTP chunk size 2018-02-04 07:33:18 +07:00
e4a60912b8 [youtube] Enforce using chunked HTTP downloading for DASH formats 2018-02-04 07:17:26 +07:00
00c97e3e7a [downloader/http] Add ability to pass downloader options via info dict 2018-02-04 07:16:22 +07:00
cf7259bc93 [downloader/http] Fix 302 infinite loops by not reusing requests 2018-02-04 06:58:34 +07:00
b54d4a5ce8 Document http_chunk_size 2018-02-04 02:53:50 +07:00
db157d2a2a release 2018.02.03 2018-02-03 23:36:03 +07:00
6fcc053947 [test_downloader_http] Use try_rm 2018-02-03 23:24:27 +07:00
a3e8146ea8 [ChangeLog] Actualize 2018-02-03 23:21:18 +07:00
f19eae429a [test_http] Use 127.0.0.1 instead of localhost 2018-02-03 23:09:25 +07:00
ba515388b8 Introduce --http-chunk-size 2018-02-03 23:08:58 +07:00
e2e18694db [redbulltv] fix extraction(closes #15481) 2018-02-03 15:43:33 +01:00
4989d351b4 flake8: Ignore E741 'ambiguous variable name'
Most of violating codes are reverse-engineered JavaScripts. IMO it's
better to keep original (obfuscated) names.

[skip ci]
2018-02-03 21:02:30 +08:00
1367c798e3 [redtube] Fix metadata extraction (closes #15472) 2018-02-02 22:32:53 +07:00
9a340af37e [compat] Mute some F821 under python 3 2018-02-02 03:18:22 +07:00
3c3bceb41d [pladform] Respect platform id and extract HLS formats (closes #15468) 2018-02-02 03:07:30 +07:00
64a12edb48 [rtlnl] Remove progressive formats (closes #15459) 2018-02-01 21:30:17 +07:00
4bf18702e6 [6play] do no modify asset urls with a token(#15248) 2018-02-01 11:22:05 +01:00
ecc218ab14 [nationalgeographic] Relax _VALID_URL 2018-01-31 09:19:11 +01:00
d6b152915c [dplay] Relax _VALID_URL (closes #15458) 2018-01-31 01:29:00 +07:00
69a934e9ad [ISSUE_TEMPLATE_tmpl.md] Add entry on checking URLs availability 2018-01-30 23:31:41 +07:00
5fa2a6a561 [soundcloud] Remove unused _IPHONE_CLIENT_ID 2018-01-30 22:51:43 +07:00
d2a422f548 [cbsinteractive] fix data extraction(closes #15451) 2018-01-29 15:33:24 +01:00
b9d52fb2ca [seznamzpravy] Remove debug output 2018-01-29 04:40:07 +07:00
466000fc6b [amcnetworks] add support for SundanceTV(closes #9260) 2018-01-28 11:30:20 +01:00
65220c3bd6 Add support for IronPython 2018-01-28 05:48:18 +07:00
c989bdbef8 [downloader/ism] Fix Python 3.2 support 2018-01-28 05:20:21 +07:00
eee1692ff3 release 2018.01.27 2018-01-27 23:44:28 +07:00
07e56e6df7 [ChangeLog] Actualize 2018-01-27 23:41:25 +07:00
3c3a07ee0b [seznamzpravy] Improve and simplify (closes #14616) 2018-01-27 23:36:44 +07:00
27940ca09c [seznamzpravy] Add extractor (closes #14102) 2018-01-27 23:34:31 +07:00
3931b84597 [extractor/common] Improve _json_ld for articles 2018-01-27 23:24:38 +07:00
a0ee342b50 [dplay] Bypass geo restriction 2018-01-26 23:56:31 +07:00
864a4576b7 [dplay] Add support for disco-api videos (closes #15396) 2018-01-26 23:49:47 +07:00
bbb7c3f7e9 [youtube] Extract precise error messages (closes #15284) 2018-01-25 22:30:33 +08:00
9d6458a206 [teachertube] Capture and output error message 2018-01-24 22:46:04 +07:00
837b061710 [teachertube] Fix and relax thumbnail extraction (closes #15403) 2018-01-24 22:41:25 +07:00
967ebbdb6c [prosiebensat1] add another clip ID regexp(fixes #15378) 2018-01-23 19:22:44 +01:00
dc400ed6a2 [tbs] update tokenizer url(fixes #15395) 2018-01-23 19:06:46 +01:00
cf2820710d Switch codebase to use compat_b64decode 2018-01-23 22:23:12 +07:00
5d7d805ca9 [mixcloud] Use compat_b64decode (closes #15394) 2018-01-23 21:53:45 +07:00
f206126df0 [compat] Add compat_b64decode 2018-01-23 21:53:01 +07:00
021bd012bb [thesixtyone] Remove extractor (closes #15341) 2018-01-22 22:30:28 +07:00
6e5eacb770 release 2018.01.21 2018-01-21 21:26:05 +07:00
d7da6db4e1 [ChangeLog] Actualize 2018-01-21 21:23:24 +07:00
721a0c3c7b [prosiebensat1] Relax clip id 2018-01-21 21:22:38 +07:00
e0ab56571e [southparkdk] Add support for southparkstudios.nu 2018-01-21 18:42:34 +07:00
99d6e696fc [southpark] Add tests for collections (closes #14803) 2018-01-21 18:41:46 +07:00
6289e07883 [southpark] Add support for collections 2018-01-21 18:41:10 +07:00
655c410063 [test_download] Fix download tests for lazy extractors (closes #13554, closes #13757) 2018-01-21 18:15:11 +07:00
b2a027fc6f [franceinter] Fix upload date extraction (closes #14996) 2018-01-21 17:50:53 +07:00
0d9c48de4f [extractor/common] Improve DASH formats extraction for jwplayer (#9242, #15187) 2018-01-21 17:42:48 +07:00
df58ecbeba [rtvs] Add extractor (closes #9242, closes #15187) 2018-01-21 17:40:23 +07:00
ac458e90a3 [restudy] Extend _VALID_URL (#15347) 2018-01-21 01:22:16 +07:00
7df18fcc65 [restudy] Fix extraction (closes #15347) 2018-01-20 23:19:02 +07:00
c707b1d828 [test_utils] Add tests for malformed JSON handling in js_to_json 2018-01-20 23:00:09 +07:00
c384d537f8 [util] Improve scientific notation handling in js_to_json (closes #14789) 2018-01-20 22:54:21 +07:00
e7f3529f68 [youtube:live] Improve live detection (closes #15365) 2018-01-20 17:57:20 +07:00
7d5406216a [springboardplatform] Add extractor 2018-01-20 00:34:10 +07:00
2a3683c378 prosiebensat1: add another clip ID regexp 2018-01-19 18:26:47 +01:00
154e4fdace [ringtv] Remove extractor (closes #15345) 2018-01-19 22:49:58 +07:00
e2fc6df169 release 2018.01.18 2018-01-18 23:41:44 +07:00
68da3d033c [ChangeLog] Actualize 2018-01-18 23:39:15 +07:00
67408fe0e9 [soundcloud] Update client id (closes #15306) 2018-01-18 22:30:43 +07:00
cad9caf76b [kamcord] Remove extractor (closes #15322) 2018-01-18 22:26:43 +07:00
4471affc34 [spiegel] Add support for nexx videos (closes #15285) 2018-01-17 22:03:56 +07:00
1370dba59f [twitch] Fix authentication and error capture (closes #14090, closes #15264) 2018-01-16 22:34:16 +07:00
1d1d60f6dd [vk] Detect more errors due to copyright complaints (#15259) 2018-01-16 00:51:50 +07:00
a86922c470 [README.md] Clarify macOS name 2018-01-14 00:58:38 +07:00
e11ccd76c6 release 2018.01.14 2018-01-14 00:13:56 +07:00
dd896a6a07 [ChangeLog] Actualize 2018-01-14 00:10:04 +07:00
391dd6f094 [youtube] Fix live streams extraction (closes #15202) 2018-01-14 00:03:22 +07:00
0ce39bc542 [wdr] Fix test 2018-01-13 23:33:52 +07:00
1915662d4f [wdr] Bypass geo restriction 2018-01-13 23:30:56 +07:00
54e8f62e01 [wdr] Rework extractors (closes #14598) 2018-01-13 23:30:25 +07:00
2d8bb80c60 [wdr:elefant] Add extractor 2018-01-13 23:29:36 +07:00
df16e645f6 [gamestar] Fix issues (closes #15179) 2018-01-13 19:38:58 +07:00
d4aedca3bd [gamestar] Add support for gamepro.de (closes #3384) 2018-01-13 19:36:59 +07:00
47e2a9bc53 [viafree] Skip rtmp formats (closes #15232) 2018-01-13 18:47:47 +07:00
e565a6386e Credit @scil for ximalaya extractor (#14687)
[ci skip]
2018-01-12 15:36:01 +08:00
609850acfb [pandoratv] Add support for mobile URLs (closes #12441) 2018-01-11 23:10:18 +07:00
64287560e4 [pandoratv] Add support for new URL format (closes #15131) 2018-01-11 23:06:56 +07:00
37941fe204 [ChangeLog] Update after #14687
[ci skip]
2018-01-11 20:36:06 +08:00
a90641fe87 [ximalaya_extractor] Add new extractor ximalaya (#14687)
* [ximalaya_extractor] Add new extractor

* format change according by flake8

* changes accoring to review by @yan12125 at github pull #14687

* change %d to %s in a temp str

* seond changes accoring to review by @yan12125 at github pull #1468

* improve TESTS about contains

* changes accoring to third review by @yan12125 at github pull #1468

* forth changes accoring to forth review by @yan12125 at github pull #1468
2018-01-11 20:35:09 +08:00
1b79daffd9 [digg] Improve extraction 2018-01-10 22:19:51 +07:00
e654829b4c [digg] Add extractor (closes #15214) 2018-01-10 21:24:22 +07:00
2b4e1ace4a [limelight] Tolerate empty pc formats (closes #15150, closes #15151, closes #15207) 2018-01-10 05:39:57 +07:00
310ea4661d [ndr:embed:base] Make separate formats extraction non fatal (closes #15203) 2018-01-09 22:04:50 +07:00
5b23845125 Credit @sprhawk for the Weibo extractor (#15079) 2018-01-09 19:35:39 +08:00
0f71de0761 [ChangeLog] Update after #15079 2018-01-09 18:13:49 +08:00
4df1098c3f Merge branch 'sprhawk-weibo' 2018-01-09 18:13:11 +08:00
5eca00a2e3 [weibo] Misc improvements 2018-01-09 18:12:55 +08:00
1dd38dc0f4 Merge branch 'weibo' of https://github.com/sprhawk/youtube-dl into sprhawk-weibo 2018-01-09 17:31:52 +08:00
8005dc68cb [ok] Add support for live streams 2018-01-08 21:53:03 +07:00
a39e15c516 [canalplus] fix extraction(closes #15072) 2018-01-07 22:15:44 +01:00
7643916a37 [ChangeLog] update after #15188
[ci skip]
2018-01-08 01:32:13 +08:00
3a513f29ad fix bilibili extraction (closes #15171) 2018-01-08 01:30:04 +08:00
950b5f2969 release 2018.01.07 2018-01-07 23:52:16 +07:00
8faa9576bb [ChangeLog] Actualize 2018-01-07 23:48:56 +07:00
b0ead0e09a [jwplatform] Add support for multiple embeds (closes #15192) 2018-01-07 21:49:23 +07:00
0a5b1295b7 [motherless:group] Relax entry extraction and add a fallback scenario 2018-01-07 00:31:53 +07:00
a133eb7764 [motherless:group] Capture leading slash of video path 2018-01-07 00:02:41 +07:00
f12628f934 [mitele] Fix extraction (closes #15186) 2018-01-06 23:58:00 +07:00
45283afdec [motherless] Add support for groups 2018-01-06 23:33:40 +07:00
b7c74c0403 [lynda] Relax _VALID_URL (closes #15185) 2018-01-06 23:12:30 +07:00
0b0870f9d0 [soundcloud] Fallback to avatar picture for thumbnail (closes #12878) 2018-01-05 08:25:42 +07:00
c2f18e1c49 [ChangeLog] Update after #15137
[skip ci]
2018-01-04 22:28:00 +08:00
da35331c6c [youku] Fix list extraction.(close #15135) (#15137)
* [youku] Fix list extraction.(close #15135)

Change-Id: I2e9c920143f4f16012252625943a8f18b8ff40eb

* [youku] Remove KeyError try-except

Change-Id: Ic46327905cbef1356b7b12d5eb3db5d9746ca338
2018-01-04 22:25:28 +08:00
de329f64ab [openload] Fix extraction (closes #15166) 2018-01-04 13:26:08 +08:00
75ba0efb52 [lynda] Skip invalid subtitles (closes #15159) 2018-01-03 16:41:28 +07:00
f0c6c2bce2 [twitch] Pass video id to url_result when extracting playlist 2018-01-03 16:22:55 +07:00
9650c3e91d [rtve.es:alacarta] Fix extraction of some new URLs 2018-01-02 21:12:39 +01:00
b5e531f31a [acast] Fix extraction 2018-01-02 23:32:17 +07:00
7a6c204fcb [travis] Add Jython build 2018-01-02 21:13:41 +07:00
d7cd9a9e84 [utils] Fix youtube-dl under PyPy3 on Windows 2018-01-01 22:48:27 +07:00
54009c246e [travis] Add PyPy builds 2018-01-01 21:54:28 +07:00
b300cda476 [YoutubeDL] Output python implementation in debug header 2018-01-01 21:52:24 +07:00
6648fd8ad6 changed to use .get to get field from json object 2018-01-01 18:33:14 +08:00
04cf1a191a release 2017.12.31 2017-12-31 04:30:49 +07:00
c95c08a856 [ChangeLog] Actualize 2017-12-31 04:28:01 +07:00
126f225bcf [extractor/common] Add container meta field for formats extracted in _parse_mpd_formats 2017-12-31 04:04:09 +07:00
4f5cf31977 [slutload] Add support for mobile URLs 2017-12-31 01:41:07 +07:00
77341dae14 [abc:iview] Improve extraction and bypass geo restriction (closes #14782) 2017-12-31 01:27:28 +07:00
2e65e7db9e [abc:iview] Fix extraction (closes #14711)
ABC dropped unmetering, so change to metered hls urls which
require auth.
2017-12-31 01:27:22 +07:00
538d4f8681 [downloader/hls] Use HTTP headers for key request 2017-12-31 01:15:35 +07:00
620ee8712e [openload] Fix extraction (closes #15118) 2017-12-30 15:03:13 +08:00
2ca7ed41fe [mediasite] Improve extraction and code style, add support for DASH (closes #11185, closes #14343, refs #5428) 2017-12-30 08:04:43 +07:00
8056c8542d [mediasite] Add extractor, subsume sandia and collegerama extractors 2017-12-30 07:23:41 +07:00
2501d41ef4 [common] use AACL as the default fourcc when AudioTag is 255 2017-12-30 07:22:07 +07:00
d97cb84b31 [ufctv] Add new extractor(closes #14520) 2017-12-30 00:30:41 +01:00
2c8e11b4af [pluralsight] Fix missing first line of subtitles (closes #11118) 2017-12-30 05:56:47 +07:00
d2c5b5a951 [openload] Fallback on f-page extraction (closes #14665, closes #14879) 2017-12-30 05:53:56 +07:00
580f3c79d5 [vimeo] Improve password protected videos extraction (closes #15114) 2017-12-30 03:54:14 +07:00
48058d82dc replace unused _download_webpage_handle with _download_webpage 2017-12-30 01:14:21 +08:00
6a41a12d29 replace split with strip_jsonp 2017-12-30 01:11:30 +08:00
5c97ec5ff5 replace urlencode.encode with urlencode_postdata 2017-12-30 01:08:56 +08:00
9d6ac71c27 [extractor/common] Fix extraction of DASH formats with the same representation id (closes #15111) 2017-12-29 23:14:56 +07:00
84f085d4bd [aws] fix canonical/signed headers generation in python 2(closes #15102) 2017-12-29 00:13:40 +01:00
a491fd0c6f release 2017.12.28 2017-12-28 23:12:56 +07:00
99277daaac [ChangeLog] Actualize 2017-12-28 23:10:42 +07:00
640788f6f4 [internazionale] Improve extraction (closes #14973) 2017-12-27 23:27:48 +07:00
1ae0f0a21d [internazionale] Add extractor 2017-12-27 23:27:43 +07:00
616bb95b28 [playtvak] Relax video regex and make description optional 2017-12-27 22:57:26 +07:00
be069839b4 [filmweb] improve extraction 2017-12-26 19:41:08 +01:00
c33de004e1 Merge branch 'master' of github.com:rg3/youtube-dl into weibo 2017-12-26 22:27:26 +08:00
42a1012c77 fix according to "https://github.com/rg3/youtube-dl/pull/15079#discussion_r158688607" 2017-12-26 22:26:01 +08:00
a14001a5a1 [Filmweb] Add extractor 2017-12-26 15:19:37 +01:00
db145ee54a [espn] Add new extractor for http://fivethirtyeight.com(closes #6864) 2017-12-26 14:20:21 +01:00
45d20488f1 [umg:de] Add new extractor(closes #11582)(closes #11584) 2017-12-26 12:32:04 +01:00
2593651224 fix compat_urllib_request for python2.7 2017-12-26 16:46:01 +08:00
951043724f re-format code to pass flake8 2017-12-26 16:38:51 +08:00
d2be5bb5af change to use compat urllib 2017-12-26 16:28:47 +08:00
447a5a710d added weibo mobile site support 2017-12-26 16:24:56 +08:00
0f897e0929 [espn] add support for espnfc and extract more formats(closes #8053) 2017-12-25 23:29:09 +01:00
173558ce96 [ChangeLog] Update after #15065 2017-12-25 22:06:18 +08:00
d3ca283235 [youku] Add test case.
Some playlist has no data-id value.

Change-Id: I97455f2907f08bda03b538cdc13ec827e2f8ce26
2017-12-25 22:02:47 +08:00
d99a1000c7 [youku] Fix list extraction.(close #15065)
Change-Id: I578fdc5b69509bdcd8d3191e3917afe47c234ff6
2017-12-25 22:02:47 +08:00
a75419586b [openload] Remove a confusing exception
If phantomjs is not installed, there's an error besides the missing
phantomjs exception:

Exception ignored in: <bound method PhantomJSwrapper.__del__ of <youtube_dl.extractor.openload.PhantomJSwrapper object at 0x7f8ad5e78278>>
Traceback (most recent call last):
  File "/home/yen/Projects/youtube-dl/youtube_dl/extractor/openload.py", line 142, in __del__
    os.remove(self._TMP_FILES[name].name)
AttributeError: 'PhantomJSwrapper' object has no attribute '_TMP_FILES'
2017-12-24 20:47:42 +08:00
273c23d960 [openload] Add support for oload.stream (closes #15070) 2017-12-24 13:53:27 +07:00
b954e72c87 [ChangeLog] typo 2017-12-23 23:42:02 +08:00
116561697d [ChangeLog] Update after #14903 2017-12-23 23:41:24 +08:00
0e25a1a278 [youku] Update ccode
Change-Id: Id397e814e81ff560506d68563b7409eebbe5943d
2017-12-23 23:34:42 +08:00
307a7588b0 release 2017.12.23 2017-12-23 21:24:18 +07:00
c2f2f8b120 [kaltura] Fix typo 2017-12-23 21:22:41 +07:00
f5a6321107 [ChangeLog] Actualize 2017-12-23 21:17:53 +07:00
69d69da98a [kaltura] Add another embed pattern for entry_id
For cases when player configuration map is setup via indexing operator, e.g. kalturaPlayerConfiguration_1_lre6rg3i_10[entry_id] = 1_lre6rg3i (see https://www.heise.de/video/artikel/odcast-c-t-uplink-20-1-Apple-CarPlay-vs-Android-Auto-Galileo-3D-Sound-erklaert-3919694.html)
2017-12-23 21:17:53 +07:00
5c5e60cff8 [voot] Fix video identification 2017-12-23 21:17:53 +07:00
2132edaa03 [extractor/common] Move X-Forwarded-For setup code into _request_webpage 2017-12-23 21:17:53 +07:00
4b7dd1705a [7plus] Add new extractor(closes #15043) 2017-12-23 13:22:20 +01:00
9e3682d555 [MANIFEST.in] Include all test data in PyPI package 2017-12-22 23:53:27 +07:00
3e191da6d9 [Makefile] Add AUTHORS to youtube-dl.tar.gz 2017-12-22 23:46:08 +07:00
963d237d26 Add LICENSE, AUTHORS and ChangeLog to PyPI package (closes #15054) 2017-12-22 23:38:16 +07:00
d2d766bc6d [animeondemand] Fix typo 2017-12-20 23:18:14 +07:00
17c3aced5d [animeondemand] Relax login error regex 2017-12-19 22:53:04 +07:00
78466fcab5 [shahid] add support for show pages(closes #7401) 2017-12-19 02:00:38 +01:00
3961c6cb9d [YoutubeDL] Add support for playlist_uploader and playlist_uploader_id in output template (closes #11427, #15018) 2017-12-19 03:53:44 +07:00
07aeced68e [youtube] Extract uploader, uploader_id and uploader_url for playlists (#11427, #15018) 2017-12-19 03:51:28 +07:00
c10c93238e [extractor/common] Introduce uploader, uploader_id and uploader_url meta fields for playlists (#11427, #15018) 2017-12-19 03:51:03 +07:00
4a109f81bc [afreecatv] Improve format extraction (closes #15019) 2017-12-19 00:38:39 +07:00
99081da90c [downloader/fragment] Encode filename of fragment being removed (closes #15020) 2017-12-18 03:31:53 +07:00
7e81010987 [cspan] add support for audio only pages and catch page errors(closes #14995) 2017-12-17 19:15:59 +01:00
549bb416f5 [mailru] Fix issues and improve (closes #14904) 2017-12-17 18:38:27 +07:00
25475dfab3 [mailru] Add support for embed URLs 2017-12-17 18:37:03 +07:00
3dfa9ec213 [crunchyroll] Future-proof XML element checks(closes #15013) 2017-12-17 09:15:44 +01:00
06dbcd7be4 [cbslocal] Fix timestamp extraction (closes #14999, closes #15000) 2017-12-16 21:57:30 +07:00
b555ae9bf1 [utils] Add another date format pattern (#14999) 2017-12-16 21:56:16 +07:00
c402e7f3a0 [discoverygo] correct ttml subtitle extension 2017-12-16 12:55:44 +01:00
498a8a4ca5 [vk] Make view count optional (closes #14979) 2017-12-15 22:53:56 +07:00
d05ba4b89e [disney] skip Apple FairPlay formats(#14982) 2017-12-15 09:28:07 +01:00
23f511f5c7 [voot] sort formats 2017-12-15 09:05:59 +01:00
1c4804ef9b [voot] fix format extraction(closes #14758) 2017-12-14 23:05:43 +01:00
8ff2b16435 release 2017.12.14 2017-12-14 05:19:21 +07:00
c6a5a811a1 [ChangeLog] Actualize 2017-12-14 05:15:51 +07:00
3fae11ac00 [itv] Improve extraction, extract more subtitles and duration (closes #14944) 2017-12-14 04:49:07 +07:00
7974e289a1 [postprocessor/xattr] Clarify NO_SPACE message (#14970) 2017-12-14 01:05:02 +07:00
6bf9c28b0a [byutv] add support for geo restricted videos 2017-12-13 17:51:56 +01:00
bec49996c6 [downloader/http] Return actual download result (closes #14971) 2017-12-13 23:49:05 +07:00
c8be7d5f74 [byutv] Fix extraction (closes #14966, closes #14967) 2017-12-13 23:14:30 +07:00
15960255fe [tbs] fix typo 2017-12-12 18:16:45 +01:00
6b2d8c9182 [bbc.co.uk] fix extraction for 320k m3u8 streams
broken since 197224b7a4
2017-12-12 18:04:05 +01:00
e6b8803d59 [toutv] add support special video urls(closes #14179) 2017-12-12 11:11:44 +01:00
cb0c2310fb [discovery] fix free videos extraction(#14157)(#14954) 2017-12-12 10:33:03 +01:00
23b6e23002 [tvnow] fix extraction(closes #7831) 2017-12-11 21:36:12 +01:00
127e98d31d [nickelodeon:br] correct extractor name 2017-12-11 17:21:31 +01:00
e4f201bc1b [extractors] add import for NickBrIE 2017-12-11 17:19:16 +01:00
08d77a95c9 [nickelodeon:br] add support for Nickelodeon(Brazil) websites(closes #14893) 2017-12-11 17:18:05 +01:00
5868079e99 [nick.com] improve extraction(fixes #14876) 2017-12-11 14:37:03 +01:00
b6f78d76c1 [tbs] fix extraction(fixes #13658) 2017-12-11 13:39:25 +01:00
0c69958844 add other properties; remove print verbose 2017-12-11 16:02:14 +08:00
3281af3464 a working version 2017-12-11 15:56:54 +08:00
29ac31afaf simply get the correct webpage, but not parsed to extract information 2017-12-11 12:26:19 +08:00
1fa0dce2c0 release 2017.12.10 2017-12-10 23:18:53 +07:00
fa1dd6d2cd [ChangeLog] Actualize 2017-12-10 23:15:24 +07:00
c38970ca10 [culturebox] Improve video id extraction (closes #14947) 2017-12-10 22:46:21 +07:00
51f2863357 [twitter] improve extraction(closes #14197) 2017-12-10 14:11:09 +01:00
913b61eeee [udemy] Extract more HLS formats 2017-12-09 20:02:54 +07:00
6f1ec339a0 [udemy] Improve course id extraction (closes #14938) 2017-12-09 20:02:49 +07:00
a3de5e6c0e [stretchinternet] Fix issues and improve (closes #14576) 2017-12-09 17:59:08 +07:00
f4cc03d60b [stretchinternet] Add extractor 2017-12-09 17:58:49 +07:00
2a57b62b80 [ellentube] Fix issues, improve and simplify (closes #14570) 2017-12-09 02:16:54 +07:00
e2707a832c [ellentube] Fix extraction (closes #14407) 2017-12-09 02:16:48 +07:00
1115271ac6 [raiplay:playlist] Fix issues and improve (closes #14563) 2017-12-09 00:48:04 +07:00
d21d0ba6c1 [raiplay:playlist] Add extractor 2017-12-09 00:47:40 +07:00
a670b1ba26 [README.md] Add is_live, start_time and end_time to output template section (closes #14926) 2017-12-07 22:16:41 +07:00
1bd4fc96e6 [sonyliv] extract higher quality formats and bypass geo restriction(closes #14922) 2017-12-07 08:46:30 +01:00
684ae10236 [fox] add support for adobe pass auth and extract subtitles(close #14489)(closes #14205) 2017-12-06 22:56:14 +01:00
3c4fbfeca2 [dailymotion] remove dailymotion cloud extractor(closes #6794)
https://web.archive.org/web/20160312110217/https://www.dmcloud.net/
2017-12-06 10:56:48 +01:00
b271e33526 [xhamster] Add support for mobile URLs and fix thumbnail extraction 2017-12-06 00:08:31 +07:00
d3f8b76b69 [extractor/generic] Fix typo (closes #14902)
Don't pass video_id as mpd_id
2017-12-05 23:11:15 +07:00
91328f26b0 [ard] Skip invalid stream URLs (closes #14906) 2017-12-05 23:01:57 +07:00
61d18c8a4b [porncom] Fix metadata extraction (closes #14911) 2017-12-05 22:42:02 +07:00
c94427dd60 [pluralsight] Detect agreement request (#14913) 2017-12-05 22:34:56 +07:00
d4f05d4731 [utils] add sami mimetype to mimetype2ext 2017-12-03 00:04:43 +01:00
d7df308981 [toutv] fix login(closes 14614) 2017-12-02 20:22:40 +01:00
0d56eddc59 release 2017.12.02 2017-12-02 21:34:34 +07:00
e25ee72657 [ChangeLog] Actualize 2017-12-02 21:29:06 +07:00
78593e294c Add references for #14844 2017-12-02 21:22:43 +07:00
593f2f7989 [downloader/fragment] Commit part file after each fragment
In order to obtain correct resume_len on next iteration
2017-12-02 21:21:11 +07:00
603fc4e0ea [extractor/common] Add durations for DASH fragments with bare SegmentURLs 2017-12-02 21:21:01 +07:00
41bf647e89 [extractor/common] Add support for DASH manifests with SegmentLists with bare SegmentURLs 2017-12-02 21:16:36 +07:00
fea92aa65d [xhamster] Fix extraction (closes #14884) 2017-12-02 19:04:59 +07:00
0981585bef [youku] Update ccode (closes #14872) 2017-12-02 18:16:22 +07:00
f5ac68d88f [mnet] fix format extraction(fixes #14883) 2017-11-30 23:45:33 +01:00
1663b32946 [xiami] add Referer header to api request 2017-11-29 20:36:55 +01:00
5ea765fb72 [mtv] correct scc extention in extracted subtitles(closes #13730) 2017-11-29 17:50:38 +01:00
fb61b57d0f [vvvvid] fix extraction for kenc videos(fixes #13406) 2017-11-29 16:09:45 +01:00
07cf18b9c5 [br] add support for BR Mediathek videos(fixes #14560)(fixes #14788) 2017-11-29 14:21:38 +01:00
5f699251e9 [daisuki] add support for motto.daisuki.com(fixes #14681) 2017-11-28 10:57:22 +01:00
a3474aa59e [Odnoklassniki] fix api metadata request(fixes #14862) 2017-11-28 09:04:51 +01:00
115afb77ec [itv] update hls formats extraction 2017-11-27 21:59:27 +01:00
53f024e7c5 [pbs] add another media id regex 2017-11-27 16:55:27 +01:00
ffe6979ef9 [utils] add hvc1 codec code to parse_codecs 2017-11-27 16:55:27 +01:00
dafb4c6647 [Makefile] Include setup.cfg in the tarball (closes #14857) 2017-11-27 22:49:35 +08:00
82a62de192 [Makefile,devscripts/run_tests.sh] Actually exclude network tests
Closes #14858
2017-11-27 21:32:06 +08:00
f58a506044 [test_InfoExtractor] Fix flake8 2017-11-27 21:30:47 +08:00
5ddeb7702a release 2017.11.26 2017-11-26 21:49:22 +07:00
6c07f0b288 [ChangeLog] Actualize 2017-11-26 21:37:27 +07:00
e94d1adc36 Add testdata to youtube-dl.tar.gz (closes #14854) 2017-11-26 21:10:32 +07:00
d08dcd2dbd [test_YoutubeDL] Fix typo (closes #14856) 2017-11-26 21:06:14 +07:00
7512aa986f Fix some only matching tests (closes #14855) 2017-11-26 20:53:10 +07:00
93f3f10cdc [fczenit] fix extraction 2017-11-25 19:28:26 +01:00
87dac57cf6 [firstpost] remove extractor 2017-11-25 18:50:15 +01:00
b485d5d6bf [nexx] make http format ids more consistent 2017-11-25 18:36:31 +01:00
a238a868ba [freespeech] fix extraction 2017-11-25 18:25:00 +01:00
c0f647a179 [nexx] extract more formats 2017-11-25 18:13:26 +01:00
6ff27b8d5a [openload] Don't use bare except when removing temp files 2017-11-26 00:05:28 +07:00
9ef909f2b2 [openload] Add support for openload.link 2017-11-26 00:04:13 +07:00
8cfbcfab9a [tnaflix] Extract common parts of tnaflix and empflix 2017-11-25 23:42:20 +07:00
b7785cf156 [empflix] Relax _VALID_URL 2017-11-25 23:42:20 +07:00
9105523818 [empflix] Fix extractrion 2017-11-25 23:42:20 +07:00
dbb25af657 [tnaflix] Don't modify download URLs (closes #14811) 2017-11-25 23:42:20 +07:00
fe4bfe36e1 [gamersyde] remove extractor 2017-11-25 15:58:28 +01:00
6f5c598a28 [france2.fr:generation-what] fix extraction 2017-11-25 15:49:49 +01:00
cd9ff4ec5b [massengeschmacktv] Add support for Massengeschmack TV(replaces Fernseh Kritik TV) 2017-11-24 20:00:01 +01:00
c6c6a64aa5 [fox9] fix extraction 2017-11-24 19:00:56 +01:00
e0a8686f48 [faz] fix extraction and add support for Perform Group embeds(fixes #14714) 2017-11-24 18:42:41 +01:00
6049176471 [ChangeLog] Update after #14828
[skip ci]
2017-11-24 21:40:51 +08:00
805f5bf759 [Generic] ie_key in JWPlatform test 72 2017-11-24 21:39:55 +08:00
32ad4f3faf [JWPlatform] Use non-capturing group in RE
Per @yan12125.
2017-11-24 21:39:55 +08:00
6899b1d9e8 [Generic] Update test 69 (suffolk/sjc)
suffolk.edu/sjc => suffolk.edu/sjc/live.php

Unfortunately it only transmits video a few mornings per month, so
leaving the 'skip' is probably appropriate. Updating the 'skip- to
include the calendar information though.
2017-11-24 21:39:55 +08:00
939be9adfe [JWPlatform] Support iframes
Support content.jwplatform... src attributes inside <iframe> tags in
addition to <script> tags. Just a regexp change.

Add a test (currently Generic_72).
2017-11-24 21:39:55 +08:00
2688664762 [culturebox] Fix extraction (closes #14827) 2017-11-23 06:39:11 +07:00
8f63941104 [youku] Fix extraction; update ccode (closes #14815) 2017-11-22 22:49:48 +08:00
a9efdf3d4a [livestream] make smil extraction non fatal(fixes #14792) 2017-11-19 12:59:31 +01:00
f610dbb05f [extractor/common] Use final URL when dumping request (closes #14769) 2017-11-18 19:04:56 +07:00
38db52adf3 [drtuber] Add support for mobile URLs 2017-11-17 01:50:07 +07:00
3192d4bc7a [spankbang] Add support for mobile URLs and fix test 2017-11-17 01:05:04 +07:00
9cbd4dda10 [instagram] Fix description, timestamp and counters extraction (closes #14755) 2017-11-15 22:14:54 +07:00
08e45b39e7 release 2017.11.15 2017-11-15 00:15:42 +07:00
fae0eb42ec [ChangeLog] Actualize 2017-11-15 00:02:54 +07:00
ea2295842f [common] skip Apple FairPlay m3u8 manifests(closes #14741) 2017-11-14 17:41:30 +01:00
a2b6aba8de [vshare] Improve extraction, fix formats sorting and carry long lines 2017-11-14 22:50:15 +07:00
ff31f2d5c3 [vshare] Capture and output error message 2017-11-14 22:39:54 +07:00
0987f2ddb2 [vshare] Fix extraction (closes #14473) 2017-11-14 22:34:45 +07:00
5871ebac47 [YoutubeDL] Fix playlist range optimization for --playlist-items (closes #14740) 2017-11-14 01:43:20 +07:00
05dee6c520 [crunchyroll] extract old rtmp formats 2017-11-13 19:15:49 +01:00
27adc9ec65 [tva] fix extraction(closes #14736) 2017-11-13 11:24:15 +01:00
388beb86e0 [gamespot] add test for #14652 2017-11-13 10:30:12 +01:00
d4e31b72b9 [gamespot] lower the preference of http formats(#14652) 2017-11-13 10:24:35 +01:00
5fc12b9549 [instagram:user] Fix extraction (closes #14699) 2017-11-12 18:36:18 +07:00
af85ce29c6 [ccma] Fix typo 2017-11-12 13:25:21 +07:00
e4d9586562 Remove sensitive data from logging in messages 2017-11-11 20:52:12 +07:00
79d1f8ed68 [gamespot] add support for article URLS(closes #14652) 2017-11-11 13:03:16 +01:00
a5203935d6 [gamespot] skip Brightcove Once http formats(#14652) 2017-11-11 13:03:16 +01:00
59d2e6d04f [cartoonnetwork] Update tokenizer_src (closes #14666) 2017-11-11 04:59:48 +07:00
a9543e37c8 [wsj] Recognize another URL pattern (closes #14704) 2017-11-11 00:29:08 +08:00
61fb07e156 [pandatv] Modernize (closes #14693) 2017-11-09 23:30:25 +07:00
4222346fb2 [pandatv] Update API URL and sign format URLs 2017-11-09 23:26:46 +07:00
cc6a960e13 use older login method(closes #11572) 2017-11-08 20:30:05 +01:00
f34b841b51 release 2017.11.06 2017-11-06 22:39:24 +07:00
e0998333fa [ChangeLog] Actualize 2017-11-06 22:36:46 +07:00
909191de91 [hotstar:playlist] Fix issues and improve (closes #12465) 2017-11-05 19:15:40 +07:00
477c97f86b [hotstar:playlist] Add extractor 2017-11-05 19:15:34 +07:00
6e71bbf4ab [hotstar] Bypass geo restriction (closes #14672) 2017-11-05 16:12:56 +07:00
181e381fda [test_InfoExtractor] Add test for #14660 2017-11-04 22:15:58 +07:00
187ee66c94 [extractor/common] Add protocol for f4m formats 2017-11-04 22:11:39 +07:00
48107c198b [f4m] Prefer baseURL for relative URLs (closes #14660) 2017-11-04 22:10:55 +07:00
cd670befc4 [22tracks] Remove extractor (closes #11024) 2017-11-02 23:48:43 +07:00
44cca168cc [skysport] add support ooyala embed_token protected videos(fixes #14641) 2017-11-02 14:16:15 +01:00
b0f4331002 [gamespot] extract formats referenced with new data fields(#14652) 2017-11-02 13:30:50 +01:00
044eeb1455 [extractor/common] Respect URL query in _extract_wowza_formats (closes #14645) 2017-11-01 23:39:26 +07:00
8fe767e072 [spankbang] Detect unavailable videos (closes #14644) 2017-10-31 23:05:25 +07:00
6d0630d880 release 2017.10.29 2017-10-29 07:22:53 +07:00
518d357b46 [ChangeLog] Actualize 2017-10-29 07:21:33 +07:00
514e8aefd4 [egghead] Fix extraction (closes #14388) 2017-10-29 07:11:37 +07:00
9211e3319e [extractor/common] Prefix format id for audio only HLS formats 2017-10-29 07:05:55 +07:00
056653bbb1 [utils] Add support for zero years and months in parse_duration 2017-10-29 07:04:48 +07:00
c3206d02e9 [fxnetworks] Extract series metadata 2017-10-29 05:20:18 +07:00
eb4b5818e2 [younow] Fix issues and improve extraction (closes #9255, closes #9432, closes #12436) 2017-10-29 04:18:43 +07:00
47a8587915 [younow] Add extractor 2017-10-29 04:17:03 +07:00
8e01f3ca81 [dctptv] Fix extraction (closes #14599) 2017-10-28 22:58:01 +07:00
f2332f18e6 [youtube] Restrict embed regex (#14600) 2017-10-27 22:26:43 +07:00
7c1f419341 [vimeo] Restrict iframe embed regex (closes #14600) 2017-10-27 22:21:47 +07:00
30e6161799 [soundgasm] Improve extraction (closes #14588) 2017-10-26 23:16:16 +07:00
dc24a7d4a2 [myvideo] Remove extractor (closes #8557)
Redirects to store.maxdome.de
2017-10-25 23:27:55 +07:00
d673ab6562 [nbc] Add support for classic-tv videos 2017-10-25 23:23:27 +07:00
b8c6ffc518 [vrtnu] Add support for cookies authentication and simplify (#11873) 2017-10-25 23:21:51 +07:00
7913e0fca7 [canvas] Add support for vrt.be/vrtnu (closes #11873) 2017-10-25 23:17:28 +07:00
cdd1ce92c4 [twitch:clips] Fix title extraction 2017-10-23 23:12:50 +07:00
55c727a547 [ndtv] Add support for sub-sites 2017-10-22 08:32:20 +07:00
36e2d3ca43 [dramafever] Fix login error message extraction 2017-10-22 08:16:30 +07:00
f7a5038305 [travis] Disable IRC notifications 2017-10-22 02:46:28 +07:00
9ff6273cae [nickru] Add support for more sites 2017-10-22 01:51:01 +07:00
f03ee0b372 [nickde] Add support for nickelodeon.be 2017-10-22 01:42:44 +07:00
cf6bda312b [nickde] Add support for nick.ch 2017-10-22 01:30:35 +07:00
3ebbd9991e [nick] Add support for more nickelodeon sites (closes #14553) 2017-10-22 01:26:58 +07:00
21ce434051 [travis] Enable IRC notifications
Let's see how is it verbose now
2017-10-21 02:14:25 +07:00
5c0e5bc4df [README.md] Add build status bagde 2017-10-21 02:11:11 +07:00
9a9de2d7b2 [travis] Allow download tests to fail and fast finish 2017-10-21 01:58:45 +07:00
424505df76 [azmedien] Fix test 2017-10-21 01:10:56 +07:00
fa3f0fd856 release 2017.10.20 2017-10-20 23:40:25 +07:00
c9dcd4b0c5 [ChangeLog] Actualize 2017-10-20 23:37:55 +07:00
fc5c47d13c [parliamentliveuk] Fix extraction (closes #14524) 2017-10-20 23:31:13 +07:00
a26a3c6d34 [soundcloud] Update client id (closes #14546) 2017-10-20 21:43:34 +07:00
382fa456ea [ChangeLog] Fix typo 2017-10-19 23:36:32 +07:00
e1d168e592 [servus] Add extractor (closes #14362) 2017-10-19 22:17:20 +07:00
ca1c9f26fa [unity] Add extractor (fixes #14528) 2017-10-19 04:46:06 +07:00
6f3b4a98c9 [downloader/fragment] Report warning instead of error on inconsistent download state 2017-10-17 22:53:34 +07:00
fa4bc6e712 [youtube] replace youtube redirect urls in description(fixes #14517) 2017-10-17 10:07:37 +00:00
6b9cbd023f [pbs] restrict direct video url regex(fixes #14519) 2017-10-17 09:23:11 +00:00
c233003afe [megaphone] Fix deprecated escape sequence 2017-10-17 15:39:06 +08:00
83fcf19e2d [drtv] Respect preference for direct http formats (#14509) 2017-10-16 05:48:45 +07:00
acc4ea6237 [eporner] Add support for embed URLs (closes #14507) 2017-10-16 05:11:25 +07:00
8cc1840ccb [arte] Capture and output error message 2017-10-15 22:12:34 +07:00
a9ee4f6e49 [downloader/hls] Fix total fragments count when ad fragments exist 2017-10-15 11:03:54 +07:00
aaab8c5e71 [niconico] Improve uploader metadata extraction robustness (closes #14135) 2017-10-15 10:40:57 +07:00
7e721e35da release 2017.10.15.1 2017-10-15 06:16:41 +07:00
bd7e1406b3 [ChangeLog] Actualize 2017-10-15 06:15:37 +07:00
74c42d9ec3 [downloader/hls] Ignore anvato ad fragments (closes #14496) 2017-10-15 06:13:48 +07:00
5efaf43c93 [downloader/fragment] Output ad fragment count 2017-10-15 06:13:07 +07:00
4827270526 [scrippsnetworks:watch] Bypass geo restriction 2017-10-15 06:11:35 +07:00
ee093a0ea0 [anvato] Add ability to bypass geo restriction 2017-10-15 06:11:02 +07:00
9bb2c7673e [redditr] Fix extraction for URLs with query (closes #14495) 2017-10-15 03:38:34 +07:00
715534083d release 2017.10.15 2017-10-15 02:26:58 +07:00
ee88c1cbc6 [ChangeLog] Actualize 2017-10-15 02:26:10 +07:00
57eb45b111 [scrippsnetworks:watch] Add support for geniuskitchen.com 2017-10-15 02:01:16 +07:00
b21ab85088 [scrippsnetworks:watch] Fix extraction (closes #14389) 2017-10-15 01:57:43 +07:00
210a2720bc [anvato] Process master m3u8 manifests
>>> Individual m3u8 manifests are not always present, e.g. anvato:anvato_scripps_app_web_prod_0837996dbe373629133857ae9eb72e740424d80a:4173834
2017-10-15 01:44:57 +07:00
685e87b61f [youtube] Fix relative URLs in description 2017-10-14 20:26:52 +07:00
c9bd503e7d [spike] bypass geo restriction 2017-10-13 08:41:57 +00:00
94a530c6cb [howstuffworks] add support for more domains 2017-10-12 19:03:47 +00:00
e650659b94 [infoq] fix http format downloading 2017-10-12 17:39:51 +00:00
2637fadc38 [generic] fix some of the tests 2017-10-12 16:14:43 +00:00
50d808f5c9 [common] add support for jwplayer youtube embeds 2017-10-12 16:12:47 +00:00
7a64c33aee [rtlnl] add support for another type of embeds 2017-10-12 16:09:06 +00:00
b0def2c297 [onionstudios] add support for bulbs-video embeds 2017-10-12 16:05:25 +00:00
81ce479f4d [udn] fix extraction 2017-10-12 16:04:41 +00:00
414e709405 [shahid] fix extraction(fixes #14448) 2017-10-12 09:20:39 +00:00
645ed3e7c9 [ChangeLog] Update after #14471
[skip ci]
2017-10-12 12:12:37 +08:00
c0bddd6d65 [kaltura] Ignore Widevine encrypted video (.wvm)
There is currently no public method to decrypt this, and there may be
other streams available that can be downloaded.

Example URL, has `.wvm` and `.mp4` formats:
https://www.voot.com/shows/bigg-boss-s11/11/538936/bigg-boss-extra-dose-arshi-s-quirky-demand/541700
2017-10-12 12:09:58 +08:00
1baba7f4a8 [vh1] Adding coding cookie 2017-10-12 12:02:26 +08:00
344d1a6794 [vh1] fix extraction(fixes #9613) 2017-10-11 20:52:14 +00:00
76581082f6 release 2017.10.12 2017-10-12 01:06:28 +07:00
2f0eb0a68a [ChangeLog] Actualize 2017-10-12 01:05:14 +07:00
7fee3377dc [steam] fix extraction(fixes #14067) 2017-10-11 17:50:08 +00:00
ff3f1a62f0 [funk] Add extractor (closes #14464) 2017-10-12 00:44:13 +07:00
694b61545c [nexx] Add support for shortcuts and relax domain id extraction 2017-10-12 00:41:20 +07:00
af0f74288d [YoutubeDL] Improve _default_format_spec (closes #14461) 2017-10-11 23:48:05 +07:00
9e38dbb19c [voxmedia] add support for recode.net(fixes #14173) 2017-10-11 15:50:20 +00:00
782195a9d4 [once] add support for vmap urls 2017-10-11 15:50:20 +00:00
26bae2d965 [generic] Add support for channel9 embeds (closes #14469) 2017-10-11 21:59:30 +07:00
5fe75f976f [tva] fix extraction(fixes #14328) 2017-10-11 14:15:52 +00:00
4fe4bda287 [tubitv] add support for new url format(fixes #14460) 2017-10-11 11:36:05 +00:00
cdab1df912 [afreecatv] remove AfreecaTVGlobalIE
the website now show this message
> Global AfreecaTV will be merged and integrated on July 20th, 2017.
Every user around the world are now able to interact with one another on
www.afreecatv.com!
2017-10-11 10:04:46 +00:00
dfc80bdd2e [ChangeLog] Update after #14420 2017-10-11 02:03:00 +08:00
04af3aca04 Remove YoutubeSharedVideoIE https://github.com/rg3/youtube-dl/issues/14303 2017-10-11 02:01:18 +08:00
d0f2d64114 [slideslive] Add extractor (closes #2680) 2017-10-10 23:45:10 +07:00
01c742ecd0 [facebook] Support thumbnails (closes #14416) 2017-10-10 23:20:38 +08:00
9e71f88105 [vvvvid] Fix typo 2017-10-10 03:48:26 +07:00
ae5af89079 [hrti:playlist] Relax _VALID_URL 2017-10-09 23:52:39 +07:00
197224b7a4 Fix some regexes 2017-10-09 23:50:53 +07:00
8992331621 [wdr] Relax media link regex (closes #14447) 2017-10-08 21:36:50 +07:00
b0dde6686c [hrti] Relax _VALID_URL 2017-10-08 05:40:08 +07:00
a22ccac1f0 [fox] Delegate to uplynk:preplay (#14147) 2017-10-08 01:34:17 +07:00
8b561bfc9d [youtube] Add support for hooktube.com (closes #14437) 2017-10-07 21:59:04 +07:00
8e751a185c release 2017.10.07 2017-10-07 05:02:53 +07:00
3fc8f5b7c2 [ChangeLog] Actualize 2017-10-07 05:01:38 +07:00
665f42d8c1 [reddit] Sort formats (closes #14430) 2017-10-07 01:40:00 +07:00
e952847541 [PULL_REQUEST_TEMPLATE.md] Add explicit entry on flake8 2017-10-07 00:58:19 +07:00
b1a7bf44b9 [lnkgo] Relax _VALID_URL 2017-10-06 23:59:09 +07:00
2e2a8e97d5 [pornflip] Extend _VALID_URL (closes #14405) 2017-10-06 23:56:31 +07:00
ac93c09ab2 [xtube] Add support for embedded URLs (closes #14417) 2017-10-06 23:53:32 +07:00
cd6fc19ed7 [YoutubeDL] Ignore duplicates in --playlist-items
E.g. '--playlist-items 2-4,3-4,3' should result in '[2,3,4]', not '[2,3,4,3,4,3]'
2017-10-06 23:50:34 +07:00
86a15ed64b [test_YoutubeDL] Add test for #14425 2017-10-06 23:41:28 +07:00
7e85e8729f [YoutubeDL] Fix out of range --playlist-items for iterable playlists and reduce code duplication (closes #14425) 2017-10-06 23:34:46 +07:00
6be08ce602 [utils] Use in OnDemandPagedList by default
Not using cache results in redundant network I/O due to downloading the same pages while using --playlist-items n-m
2017-10-06 23:13:53 +07:00
cf5f6ed5be [xvideos] Add support for embed URLs and improve extraction (closes #14409) 2017-10-05 00:27:24 +07:00
6b46285e85 [comedycentral] new shortcut :theopposition for "The Opposition" show 2017-10-04 07:45:13 +02:00
6e736d86e7 [beeg] Fix extraction (closes #14403) 2017-10-04 04:27:42 +07:00
M.K
c110944fa2 [extractor/common] Fix typo in _parse_mpd_formats 2017-10-04 03:50:27 +07:00
9524dca3ac [README.md] Use revision bound link to YoutubeDL options (closes #14401) 2017-10-04 02:53:20 +07:00
3e4cedf9e8 [tvn24] Relax _VALID_URL 2017-10-03 23:28:13 +07:00
bfd484ccff Merge pull request #14392 from snipem/nbc-fix
Fix for JSON meta data download(closes #13651)
2017-10-03 14:49:55 +00:00
b7e14f06a4 Fix for JSON meta data download
Added fixes according to #13651 and user @remitamine
2017-10-03 15:17:28 +02:00
d2ae7e24e5 [postprocessor/ffmpeg] Convert to opus using libopus (closes #14381) 2017-10-02 04:43:25 +07:00
544ffb7790 [ketnet] Add support for videos without direct sources (closes #14377) 2017-10-02 04:15:12 +07:00
117589dfa2 [canvas] Generalize mediazone.vrt.be extractor and rework canvas and een 2017-10-02 04:14:36 +07:00
839728f5bf [afreecatv] Add support for adult videos (closes #14376) 2017-10-02 03:28:25 +07:00
fcdd37d053 release 2017.10.01 2017-10-01 21:54:11 +07:00
1dd126180e [ChangeLog] Actualize 2017-10-01 21:45:56 +07:00
4e599194d6 [tvp] Add support for new URL schema (closes #14368) 2017-10-01 18:59:00 +07:00
c5b7014a9c [generic] Add support for single format Video.js embeds (closes #14371) 2017-10-01 07:01:42 +07:00
c8da40d834 [yahoo] Bypass geo restriction for brightcove (#14210) 2017-10-01 04:49:27 +07:00
b69ca0ccfc [yahoo] Use extracted brightcove account id (closes #14210) 2017-10-01 04:37:42 +07:00
2c53bd51c6 [rtve:alacarta] Fix extraction (closes #14290) 2017-10-01 03:21:17 +07:00
3836b02ce8 [YoutubeDL] PEP 8 2017-09-30 22:56:40 +07:00
fa3fdeb41f [yahoo] Fix some tests 2017-09-30 22:54:22 +07:00
eb9a15be60 [yahoo] Add support for custom brigthcove embeds (closes #14210) 2017-09-30 22:47:03 +07:00
3600fd591d [YoutubeDL] Document youtube_include_dash_manifest 2017-09-28 00:46:48 +07:00
63d990d285 [generic] Add support for Video.js embeds 2017-09-28 00:37:30 +07:00
b14b2283a0 [gfycat] Add support for /gifs/detail URLs (closes #14322) 2017-09-27 22:48:47 +07:00
02d01e15f1 [generic] Fix infinite recursion for twitter:player URLs (closes #14339) 2017-09-26 21:47:18 +07:00
db96252831 [xhamsterembed] Fix extraction (closes #14308) 2017-09-24 19:23:08 +07:00
8b389f7e3c Credit the author of multiple generic HTML5 embeds fix 2017-09-24 18:21:38 +07:00
9fc41bcb6b release 2017.09.24 2017-09-24 00:22:50 +07:00
10cab6613f [ChangeLog] Actualize 2017-09-24 00:21:34 +07:00
4d182955a2 [kakao] Fix _VALID_URL 2017-09-24 00:19:27 +07:00
011da618bd [openload] Fix _load_cookies for python 2.6 2017-09-24 00:12:40 +07:00
4c54b89e03 Hide experimental phantomjs wrapper 2017-09-24 00:08:27 +07:00
a87d7b4953 Credit @nbppp2 for americastestkitchen (#13996) 2017-09-23 23:27:28 +07:00
2f3933aa1e Credit @ishitatsuyuki for mixcloud fix (#14132) 2017-09-23 23:26:35 +07:00
aab20aabfc Credit @jdong92 for voot (#14059) 2017-09-23 23:23:27 +07:00
16f54d0751 Credit @codeasashu for voot (#11814) 2017-09-23 23:20:20 +07:00
07d1344c85 Credit @coreynicholson for vlive:playlist (#13613) 2017-09-23 23:16:27 +07:00
47b5dfb047 Credit @luboss for joj (#13268) 2017-09-23 23:14:41 +07:00
e3440d824a [24video] Fix timestamp extraction and make non fatal (#14295) 2017-09-23 07:46:53 +07:00
136507b39a [24video] Add support for 24video.adult (closes #14295) 2017-09-23 07:41:22 +07:00
7f4921b38d [heise] PEP 8 2017-09-23 07:28:29 +07:00
f70ddd4aeb [kakao] Improve (closes #14007) 2017-09-23 07:28:24 +07:00
1c22d7a7f3 [kakao] Add extractor (closes #12298) 2017-09-23 07:28:19 +07:00
5c1452e8f1 [twitter] Add support for user_id-less URLs (closes #14270) 2017-09-23 06:38:09 +07:00
4bb58fa118 [americastestkitchen] Improve (closes #13996) 2017-09-23 06:29:20 +07:00
13de91c9e9 [americastestkitchen] Add extractor (closes #10764) 2017-09-23 06:29:07 +07:00
9ce1ac4046 [generic] Fix support for multiple HTML5 videos on one page (closes #14080) 2017-09-23 05:49:48 +07:00
095774e591 [mixcloud] Improve and simplify (closes #14132) 2017-09-23 05:37:03 +07:00
2384f5a64e [mixcloud] Fix extraction (closes #14088) 2017-09-23 05:36:57 +07:00
8c2895305d [options] Accept lrc as a subtitle conversion target format (closes #14292) 2017-09-23 02:30:03 +08:00
8c6919e433 [lynda] Add support for educourse.ga (closes #14286) 2017-09-21 23:00:35 +07:00
f6ff52b473 [beeg] Fix extraction (closes #14275) 2017-09-21 04:05:33 +07:00
12ea5c79fb [nbcsports:vplayer] Correct theplatform URL (closes #13873) 2017-09-21 02:53:06 +07:00
3b65a6fbf3 [twitter] Fix duration extraction 2017-09-20 03:58:06 +07:00
dc76eef092 [tvplay] Bypass geo restriction 2017-09-20 00:00:04 +07:00
8a1a60d173 [devscripts/check-porn] Fix gettestcases import 2017-09-19 22:51:20 +07:00
4d8c4b46d5 [heise] Add support for YouTube embeds 2017-09-17 22:46:52 +07:00
9c2a17f2ce [popcorntv] Add extractor (closes #5914, closes #14211) 2017-09-17 22:19:57 +07:00
4ed2d7b7d1 Fix flake8 issues after #14225 2017-09-17 13:53:04 +08:00
8251af63a1 [viki] Update app data (closes #14181) 2017-09-16 22:45:23 +07:00
790d379e4d [morningstar] Relax _VALID_URL 2017-09-16 22:39:46 +07:00
3869028ffb [utils] Use bytes-like objects in dfxp2srt
This fixes handling of non-UTF8 TTML subtitles

Closes #14191
2017-09-16 12:18:38 +08:00
68d43a61b5 Ignore TTML subtitles 2017-09-16 12:14:48 +08:00
a88d461dff Merge pull request #14225 from Tithen-Firion/openload-phantomjs-method
Openload phantomjs method
2017-09-16 02:28:28 +08:00
a4245acef8 [noovo] Fix extraction (closes #14214) 2017-09-15 23:12:19 +07:00
6be44a50ed [dailymotion:playlist] Relax _VALID_URL (closes #14219) 2017-09-15 22:25:38 +07:00
b763e1d68c [twitch] Add support for go.twitch.tv URLs (closes #14215) 2017-09-15 22:18:38 +07:00
cbf85239bb [vgtv] Relax _VALID_URL (closes #14223) 2017-09-15 22:13:30 +07:00
159d304a9f release 2017.09.15 2017-09-15 21:48:06 +07:00
86e55e317c [ChangeLog] Actualize 2017-09-15 21:45:18 +07:00
c46680fb2a [condenast] Fix extraction (closes #14196, closes #14207) 2017-09-15 02:01:17 +07:00
fad9fc537d [tv4] fix a test URL 2017-09-14 20:47:23 +02:00
0732a90579 [orf] Add new extractor for f4m stories 2017-09-14 20:37:46 +02:00
319fc70676 [tv4] Relax _VALID_URL (closes #14206) 2017-09-14 23:50:19 +07:00
e7c3e33456 [downloader/fragment] Restart inconsistent incomplete fragment downloads (#13731) 2017-09-14 23:19:53 +07:00
757984af90 Merge pull request #12909 from remitamine/raw-sub
[YoutubeDL] write raw subtitle files
2017-09-13 17:36:40 +08:00
2f483758bc [animeondemand] Improve and modernize 2017-09-11 04:32:35 +07:00
018cc61549 [animeondemand] Bypass geo restriction 2017-09-11 04:23:42 +07:00
2709d9fa28 [animeondemand] Add support for flash videos (closes #9944) 2017-09-11 04:23:42 +07:00
7dacceae75 release 2017.09.11 2017-09-11 03:30:33 +07:00
43df248f10 [ChangeLog] Actualize 2017-09-11 03:27:43 +07:00
f12a6e88b2 [rutube:playlist] Fix suitable (closes #14166) 2017-09-11 03:23:00 +07:00
806498cf2f release 2017.09.10 2017-09-10 22:16:55 +07:00
b98339b54b [ChangeLog] Actualize 2017-09-10 22:15:55 +07:00
bf6ec2fea9 [fox] Fix extraction (#14147) 2017-09-10 22:08:32 +07:00
c3dd44e085 [rutube] Use bool_or_none 2017-09-10 19:09:27 +07:00
c7e327c4d4 [utils] Introduce bool_or_none 2017-09-10 19:08:39 +07:00
48b813748d [rutube] Rework and generalize playlist extractors (closes #13565) 2017-09-10 18:40:33 +07:00
debed8d759 [rutube:playlist] Add extractor (closes #13534) 2017-09-10 18:40:33 +07:00
51aee72d16 [README.md] Clarify how to run extractor specific test cases 2017-09-08 22:13:17 +07:00
931edb2ada [radiocanada] Add fallback for title extraction 2017-09-08 21:53:24 +07:00
5113b69124 [abcnews,chilloutsoze,cracked,vice,vk] Use dedicated YouTube embeds extraction routines 2017-09-06 00:50:25 +07:00
66c9fa36c1 [youtube] Separate methods for embeds extraction 2017-09-06 00:48:37 +07:00
c5c9bf0c12 [YoutubeDL] Ensure dir existence for each requested format (closes #14116) 2017-09-05 23:31:34 +07:00
880fa66f4f [redtube] Fix formats extraction (closes #14122) 2017-09-05 22:45:49 +07:00
6348671c4a [arte] Relax unavailability check (closes #14112) 2017-09-04 23:08:40 +07:00
efc57145c1 [manyvids] Improve (closes #14059) 2017-09-03 17:32:23 +07:00
e9b865267a [manyvids] Add support for preview videos (closes #14053) 2017-09-03 17:31:53 +07:00
bc35f07537 [vidme:user] Make tests only matching (closes #14054) 2017-09-03 17:03:51 +07:00
0b4a8eb3ac [vidme:user] Relax _VALID_URLs 2017-09-03 17:03:45 +07:00
c1c1585b31 [bpb] Improve (closes #14086) 2017-09-03 16:43:33 +07:00
0cbb841ba9 [bpb] Fix extraction (closes #14043) 2017-09-03 16:39:12 +07:00
d7c7100e3d [soundcloud] Simplify and add test (closes #14093) 2017-09-03 16:29:58 +07:00
73602bcd0c [soundcloud] Fix download URL with private tracks 2017-09-03 16:28:34 +07:00
23b2df82c7 [aliexpress:live] Fix issues (closes #13698, closes #13707) 2017-09-03 16:05:31 +07:00
503115540d [aliexpress:live] Add extractor 2017-09-03 16:05:00 +07:00
64f0e30b93 [viidea] Capture and output lecture error message (#14099) 2017-09-02 15:44:49 +07:00
a3431e1224 [radiocanada] Skip unsupported platforms (closes #14100) 2017-09-02 15:33:54 +07:00
a2022b0c40 release 2017.09.02 2017-09-02 01:08:32 +07:00
8681ed7fc8 [ChangeLog] Actualize 2017-09-02 01:04:22 +07:00
8d81f3e36d [youtube] Force old layout for each webpage (closes #14083) 2017-09-02 00:58:19 +07:00
7998520933 [youtube] Fix upload date extraction (closes #14065) 2017-08-31 00:47:58 +07:00
5b4bfbfc3b [charlierose] Add support for episodes (closes #14062) 2017-08-30 23:50:33 +07:00
53647dfd0a [bbccouk] Add support for w-prefixed ids (closes #14056) 2017-08-30 05:27:56 +07:00
22f65a9efc Merge pull request #14048 from ryandesign/patch-1
Fix build failures with old cp and zip
2017-08-28 11:22:27 +08:00
c75c384fb6 Fix build failures with old cp and zip 2017-08-27 18:07:09 -05:00
1b41da488d [googledrive] Extend _VALID_URL (closes #9785) 2017-08-28 00:50:41 +07:00
fea82c1780 [googledrive] Add support for source format (closes #14046) 2017-08-28 00:39:22 +07:00
3902cdd0e3 [pornhd] Fix extraction (closes #14005) 2017-08-27 22:37:26 +07:00
2cfa7cbdd0 release 2017.08.27.1 2017-08-27 06:09:29 +07:00
cc0412ef91 [ChangeLog] Actualize 2017-08-27 06:06:49 +07:00
1c9c8de29e [youtube] Fix extraction with --youtube-skip-dash-manifest enabled (closes #14037) 2017-08-27 06:06:39 +07:00
f031b76065 release 2017.08.27 2017-08-27 04:28:04 +07:00
62c06c593d [ChangeLog] Actualize 2017-08-27 04:27:19 +07:00
ff17be3ac9 [extractor/generic] Extract from LD-JSON last of all
Previous sources may contain several formats, e.g. http://tamasha.com/v/PgGZ
2017-08-27 03:31:40 +07:00
1ed4549942 [extractor/common] Extract format id from label attribute of source tag for HTML5 videos (#14034) 2017-08-27 03:27:05 +07:00
dd121cc1ca [extractor/common] Extract height from res attribute of source tag for HTML5 videos (closes #14034) 2017-08-27 03:12:56 +07:00
a3c3a1e128 [http] Rework HTTP downloader
* Simplify code and split into separate routines to facilitate maintaining
* Make retry mechanism work on errors during actual download not only during connection establishment phase
* Retry on ECONNRESET and ETIMEDOUT during reading data from network
* Retry on content too short and various timeout errors
* Show error description on retry
* Closes #506, closes #809, closes #2849, closes #4240, closes #6023, closes #8625, closes #9483
2017-08-27 02:22:30 +07:00
085d9dd9be [rai] Fix audio formats extraction (closes #14024) 2017-08-26 22:02:49 +07:00
151978f38a [mixcloud] Fix extraction (closes #14020) 2017-08-26 19:32:57 +07:00
c7121fa7b8 [youtube] Fix controversy videos extraction (closes #14027, closes #14029) 2017-08-26 15:38:38 +07:00
745968bc72 [mixcloud] Fix extraction (closes #14015) 2017-08-24 22:28:44 +07:00
df235dbba8 release 2017.08.23 2017-08-23 23:23:13 +07:00
c4bdc68113 [ChangeLog] Actualize 2017-08-23 23:21:19 +07:00
5bae33485c [toutv] PEP 8 2017-08-23 22:50:00 +07:00
0830f3e048 [cbc:watch] Bypass geo-restriction (closes #13993) 2017-08-23 22:45:45 +07:00
8d7a24aff6 [toutv] Relax DRM check (closes #13994) 2017-08-23 22:28:09 +07:00
37d9af306a [googledrive] Simplify and carry long lines (#13638) 2017-08-23 00:33:53 +07:00
e01c3d2ef7 [extractor/common] Introduce _parse_xml 2017-08-23 00:32:41 +07:00
05915e379a [googledrive] Add support for subtitles (fixes #13619) 2017-08-22 23:48:59 +07:00
7b67b60773 Merge pull request #13669 from bmwiedemann/master
[build] Override timestamps in zip file
2017-08-22 21:51:20 +08:00
8d9c2a681a [pornhub] Relax uploader regex (closes #13906, closes #13975) 2017-08-21 23:06:27 +07:00
903d4d1625 [README.md] Switch to HTTPS URLs 2017-08-20 23:35:39 +07:00
8239c6791a [bandcamp:album] Extract track titles 2017-08-20 23:32:33 +07:00
b359e977b9 [extractor/common] Make HLS and DASH extraction non fatal in _parse_html5_media_entries (closes #13970) 2017-08-20 14:16:58 +07:00
305d99f0bd [build] Override timestamps in zip file
to make build reproducible.
See https://reproducible-builds.org/ for why this is good

Copying files to not interfere with freshness detection.
2017-08-19 21:43:48 +02:00
d3d45e0a45 [bbccouk] Add support for events URLs (closes #13893) 2017-08-19 23:54:15 +07:00
381ad4f309 [liveleak] Support multi-video pages (closes #6542) 2017-08-19 22:48:00 +08:00
e2481b9b6e [ChangeLog] Fix 2017-08-19 22:28:58 +08:00
09747ba766 [liveleak] Support another liveleak embedding pattern (closes #13336) 2017-08-19 22:28:13 +08:00
f8f18f332f [cda] Fix extraction (closes #13935) 2017-08-19 21:44:47 +08:00
95f3f7c20a [utils] Fix unescapeHTML for misformed string like "&a&quot;" (#13935) 2017-08-19 21:40:53 +08:00
f5469da9e6 [laola1tv] Add support for tv.ittf.com (closes #13965) 2017-08-19 19:48:20 +07:00
d14d9d8903 [mixcloud] Fix extraction (closes #13958) 2017-08-18 23:31:42 +07:00
ea004d34f8 release 2017.08.18 2017-08-18 01:05:27 +07:00
2738965d98 [ChangeLog] Actualize 2017-08-18 01:03:20 +07:00
4a91910365 [qqmusic:toplist] PEP 8 2017-08-18 01:00:07 +07:00
c0892b2b46 [arte] Detect unavailable videos (closes #13945) 2017-08-18 00:58:23 +07:00
a5ac0c4755 [YoutubeDL] Sanitize byte string format URLs (#13951) 2017-08-17 23:59:12 +07:00
5551d7714d [generic] Convert redirect URLs to unicode strings (closes #13951) 2017-08-17 23:58:01 +07:00
5f5c7b92dd [udemy] Fix paid course detection (#13943) 2017-08-17 23:14:46 +07:00
93d0583e34 [pluralsight] Use RPC API for course extraction (closes #13937) 2017-08-17 22:45:40 +07:00
5d28169747 Credit Genki Sky for clippit (bfabd17b33) 2017-08-17 21:21:17 +08:00
7ddab7742c [ChangeLog] Add an entry for Genki Sky's patch 2017-08-17 16:56:37 +08:00
bfabd17b33 Add new extractor 2017-08-17 16:56:06 +08:00
12f5304556 [ChangeLog] Add entry for #13805 2017-08-17 16:40:56 +08:00
25a6e769a1 [qqmusic] Fix tests and cleanup 2017-08-17 16:39:57 +08:00
d22b67f356 Merge pull request #13805 from gam2046/master
Fix QQ Music url changed
2017-08-17 16:11:35 +08:00
a1aa659662 [periscope] Renew HLS extraction (closes #13917) 2017-08-16 23:03:42 +07:00
4850478543 [extractor/common] Add support for float durations in _parse_mpd_formats (closes #13919) 2017-08-15 23:58:00 +07:00
134d85a7bd [qqmusic] review 2017-08-15 13:14:35 +08:00
5c037c0d1f [qqmusic]support QQMusicSingerIE 2017-08-15 13:14:35 +08:00
5d1bd3b907 [qqmusic]update valid url 2017-08-15 13:14:34 +08:00
19ada898dc fix QQ Music Url changed 2017-08-15 13:14:34 +08:00
da20951a57 [mixcloud] Extract decrypt key 2017-08-14 22:39:05 +07:00
16393d6535 release 2017.08.13 2017-08-13 08:58:30 +07:00
4f049e4aa8 [ChangeLog] Actualize 2017-08-13 08:00:15 +07:00
475bcb225f [pornhub:playlistbase] Skip videos from drop-down menu for all playlists (closes #12819, closes #13902) 2017-08-13 07:53:02 +07:00
b3c6515365 [fourtube] Add support for other sites (closes #6022, closes #7859, closes #13901) 2017-08-13 07:23:29 +07:00
eb02940cc7 [generic] Add test for #13895 2017-08-13 01:11:27 +07:00
4ef9152428 [limelight] Improve embeds detection (closes #13895) 2017-08-13 00:58:39 +07:00
0c43a481b9 [reddit] Add extractors (closes #13847) 2017-08-12 23:43:51 +07:00
868f79db41 [extractor/common] Fix _media_formats 2017-08-12 19:24:26 +07:00
70851a95c3 [aparat] Extract all formats (closes #13887) 2017-08-12 17:18:23 +07:00
e74e3b63e3 [YoutubeDL] Make sure format id is not empty 2017-08-12 17:14:11 +07:00
ac8491fcca [extractor/common] Make _family_friendly_search optional 2017-08-12 17:11:35 +07:00
82889d4ae5 [extractor/common] Respect source's type attribute for HTML5 media (closes #13892) 2017-08-12 16:48:11 +07:00
92a5c41532 [mixcloud] Fix play info decryption (closes #13885) 2017-08-12 16:30:50 +07:00
1663bd6e1c [generic] Replace vzaar embed test 2017-08-11 22:02:00 +07:00
41918eaa5c [generic] Add support for vzaar embeds 2017-08-11 22:00:39 +07:00
6ed99754bb release 2017.08.09 2017-08-09 23:52:22 +07:00
0e7dfa7d16 [ChangeLog] Actualize 2017-08-09 23:49:53 +07:00
baba5f4d1d [xxxymovies] Fix title extraction (closes #13868) 2017-08-09 23:46:49 +07:00
dee04d24a4 [nick] Add support for nick.com.pl (closes #13860) 2017-08-09 23:12:02 +07:00
5b3ddadcc3 [mixcloud] Fix play info decryption (closes #13867) 2017-08-09 22:55:13 +07:00
5b232f46dc [utils] Skip missing params in cli_bool_option (closes #13865) 2017-08-09 22:28:19 +07:00
4bf22f7a10 [20min] Fix embeds extraction 2017-08-08 05:41:38 +07:00
15d1e8a23d [dplayit] Fix extraction (closes #13851) 2017-08-07 22:43:42 +07:00
ee6a611665 [niconico] Support videos with multiple formats (closes #13522) 2017-08-07 00:19:46 +08:00
463e7216c8 [niconico] Support HTML5-only videos (closes #13806) 2017-08-06 23:07:28 +08:00
903a183b6a release 2017.08.06 2017-08-06 09:05:36 +07:00
92740e4241 [ChangeLog] Actualize 2017-08-06 09:02:14 +07:00
fac188c695 [pluralsight] Fix format selection 2017-08-06 08:44:28 +07:00
16afce174e [mpora] Remove extractor (closes #13826) 2017-08-06 08:18:16 +07:00
e2b4808fd8 [voot] Improve extraction (#10255, closes #11814) 2017-08-06 08:05:29 +07:00
daaaf5f594 [voot] Add extractor 2017-08-06 08:05:24 +07:00
f172c86dcd [vlive:channel] Limit number of videos per page to 100 (closes #13830) 2017-08-05 21:17:55 +07:00
1d5472290f [podomatic] Extend _VALID_URL (closes #13827) 2017-08-05 08:28:12 +07:00
c983cc3b71 [cinchcast] Extend _VALID_URL 2017-08-05 08:17:01 +07:00
1141e9104b Use relative paths for DASH fragments (closes #12990)
10x reduced JSON size
refs #13810
2017-08-05 07:40:29 +07:00
8519b88f67 [yandexdisk] Relax _VALID_URL (closes #13824) 2017-08-05 00:59:07 +07:00
bbbe1cebfc [mlb] Update test (closes #13777) 2017-08-05 00:09:36 +07:00
f31fd0693b [vidme] Extract DASH and HLS formats 2017-08-05 00:00:21 +07:00
799802f368 [teamfour] Remove extractor (closes #13782)
Now covered with generic extractor
2017-08-04 23:54:28 +07:00
b3b5870cba [pornhd] Fix extraction (closes #13783) 2017-08-04 23:51:03 +07:00
57a38a38c3 [udemy] Fix subtitles extraction (closes #13812) 2017-08-04 23:45:13 +07:00
11a6793f80 [mlb] Extend _VALID_URL (closes #13740) 2017-08-04 22:46:54 +07:00
1f03fef994 [README.md] Improve grammar 2017-08-04 22:43:44 +07:00
183062a4ab [pbs] Add support for new URL schema (closes #13801) 2017-08-03 23:19:59 +07:00
feee8d32e4 [phantomjs] add exe version to debug info 2017-08-03 14:17:25 +02:00
8cda78ef72 [test_YoutubeDL] Add a test for #10083 2017-08-02 23:12:34 +07:00
9118c9f18a [nrktv] Update API host (closes #13796) 2017-08-01 05:21:00 +07:00
5c9ea67bc0 release 2017.07.30.1 2017-07-30 20:47:31 +07:00
f701827e31 [ChangeLog] Actualize 2017-07-30 19:43:09 +07:00
8b9f50d7cb [watchbox] Add extractor (#13739) 2017-07-30 19:09:44 +07:00
0ed4758023 [clipfish] Remove extractor 2017-07-30 19:08:44 +07:00
a0a477b885 [youjizz] Fix extraction (closes #13744) 2017-07-30 15:48:22 +07:00
198d4cb40c [generic] Add support for another ooyala embed pattern (closes #13727) 2017-07-30 01:30:04 +07:00
ca127ab2c1 [ard] Add support for lives (closes #13771) 2017-07-29 23:07:28 +07:00
e445850e69 [soundcloud] Update client id 2017-07-29 18:45:57 +07:00
836ef26486 [soundcloud:trackstation] Add extractor (closes #13733) 2017-07-29 18:41:42 +07:00
c04017519d [svtplay] Use geo verification proxy for API request 2017-07-29 15:30:53 +07:00
2a7a823211 [svtplay] Update API URL (closes #13767) 2017-07-29 15:25:32 +07:00
95908ce453 [extractor/generic] PEP 8 2017-07-29 15:13:12 +07:00
cbbe66635f [yandexdisk] Add extractor (closes #13755) 2017-07-29 15:10:19 +07:00
c5a49ff084 [downloader/hls] Use redirect URL as manifest base (#13755) 2017-07-29 15:02:41 +07:00
24e966e8da [megaphone] Add extractor 2017-07-28 12:13:19 +02:00
9682666bda [amcnetworks] Make rating optional (closes #12453) 2017-07-27 02:04:51 +07:00
f9c48d895b [cloudy] Fix extraction (closes #13737) 2017-07-26 23:12:43 +07:00
c99d6890cb [nickru] Add extractor 2017-07-23 21:02:06 +07:00
70bfab0e9a [mtv] Improve thumbnal extraction 2017-07-23 21:02:06 +07:00
f0e31e32c9 [nick] Automate geo-restriction bypass (#13711) 2017-07-23 20:40:04 +07:00
3150976669 [ISSUE_TEMPLATE_tmpl.md] Minor improvements 2017-07-23 20:33:18 +07:00
e3ce912c3d [niconico] improve error reporting (#13696) 2017-07-23 16:25:30 +08:00
73095e013f [options] Typo 2017-07-23 16:24:18 +08:00
905d18a7aa [options] Correctly hide login info from debug outputs (#13696)
Iterate over opts instead of PRIVATE_OPTS for both performance and
correctness
2017-07-23 16:22:14 +08:00
0db492c02a release 2017.07.23 2017-07-23 01:09:09 +07:00
425f41319a [ChangeLog] Actualize 2017-07-23 01:06:08 +07:00
71dde5eecf [itv] Fix production id extraction (closes #13671) 2017-07-23 00:59:07 +07:00
935d6c20c0 [vidio] Make duration non fatal and fix typo 2017-07-23 00:44:50 +07:00
e0f1fb0a27 [mtv] Skip missing video parts (closes #13690) 2017-07-23 00:25:23 +07:00
0017d9ad6d [YoutubeDL] Improve default format specification (closes #13704) 2017-07-23 00:12:01 +07:00
327c8364f1 [sportbox:embed] Fix extraction 2017-07-22 21:35:14 +07:00
359aa2fdd1 [npo] Add support for npo3.nl URLs 2017-07-22 19:15:55 +07:00
f76c02c87b [dramafever] Fix tests 2017-07-22 11:41:40 +07:00
7d9a1db111 [dramafever] Remove video id from title (closes #13699) 2017-07-22 11:40:46 +07:00
0396806f67 [YoutubeDL] Do not override id, extractor and extractor_key in url_transparent
All these meta fields must be borrowed from final extractor that actually performs extraction.
This commit fixes extractor id in download archives for url_transparent downloads. Previously, 'transparent' extractor was erroneously
used for extractor archive id, e.g. 'eggheadlesson 4n8ugwwj5t' instead of 'wistia 4n8ugwwj5t'.
2017-07-21 00:13:32 +07:00
dc6520aa3d [egghead:lesson] Add extractor (#6635) 2017-07-20 23:22:36 +07:00
c653326a14 [funnyordie] Extract more metadata (closes #13677) 2017-07-20 22:50:56 +07:00
3fcf346ac1 [youku:show] Refine playlist extraction
Handle playlists that the initial page is not the first page
2017-07-20 23:20:46 +08:00
fa63cf6c23 [youku:show] Fix playlist extraction (closes #13248) 2017-07-20 22:57:51 +08:00
85f5a74b6c [tbs] Mark as broken and skip invalid tests 2017-07-20 21:19:09 +08:00
d20b1c6725 [dispeak] Recognize sevt subdomain (closes #13276) 2017-07-20 18:14:14 +08:00
bb176df3bb [spiegel:article] Move test 2017-07-17 22:19:40 +07:00
83d00044c1 [adn] Improve error reporting (#13663) 2017-07-16 20:50:32 +07:00
7abed4e06c [crunchyroll] Relax series and season regex (closes #13659) 2017-07-16 12:40:45 +07:00
13eb526f11 [nexx:embed] PEP 8 2017-07-16 05:23:19 +07:00
00d06e3cfc [spiegel:article] Add support for nexx iframe embeds (closes #13029) 2017-07-16 04:38:20 +07:00
749ca5eced [extractor/common] Fix playlist_from_matches 2017-07-16 04:33:14 +07:00
3f59b0154a [nexx:embed] Add extractor for iframe embeds 2017-07-16 04:32:37 +07:00
089b97cfee [nexx] Improve JS embed extraction 2017-07-16 04:30:48 +07:00
decf86044d [pearvideo] Improve (closes #13031) 2017-07-16 03:06:04 +07:00
94b817edeb [pearvideo] Add extractor 2017-07-16 03:02:31 +07:00
cea931a9e5 release 2017.07.15 2017-07-15 07:36:05 +07:00
ef78563e9c [ChangeLog] Actualize 2017-07-15 07:33:26 +07:00
961ea474b6 [YoutubeDL] PEP 8 2017-07-15 07:02:57 +07:00
ea3f20494f [youtube] PEP 8 2017-07-15 07:02:57 +07:00
c7604d79e9 [spiegeltv] Delegate extraction to nexx (closes #13159) 2017-07-15 07:02:57 +07:00
4e826cd9ae [nexx] Add extractor (closes #10807, closes #13465) 2017-07-15 07:02:57 +07:00
2583c0b54e Fix bugs caused by typos 2017-07-14 23:08:32 +07:00
7d02dcfaa2 [youtube] Don't capture YouTube Red ad for creator meta field (closes #13621) 2017-07-14 22:37:04 +07:00
00dbdfc1f7 [slideshare] Fix extraction 2017-07-14 22:11:07 +07:00
f354d84807 [5tv] Add another video URL pattern (closes #13354) 2017-07-14 22:10:17 +07:00
15da37c7dc [YoutubeDL] Don't expand env variables in meta fields (closes #13637) 2017-07-14 00:42:12 +07:00
9a0942ad55 [drtv] Make HLS and HDS extraction non fatal 2017-07-11 22:59:56 +07:00
f2bb33a986 [ted] Fix subtitles extraction (closes #13628, closes #13629) 2017-07-11 21:36:45 +07:00
3615bfe1b4 [twitter] Fix remaining tests 2017-07-11 16:46:37 +08:00
e8f20ffa03 [vine] Make sure the title won't be empty
And fix a relevant TwitterCard test case
2017-07-11 16:05:15 +08:00
9be31e771c [twitter] Support HLS streams in vmap URLs 2017-07-11 15:48:48 +08:00
7f176ac477 [periscope] Support pscp.tv URLs in embedded frames
And fix a relevant twitter test
2017-07-11 15:35:19 +08:00
2edfd745df [twitter] Extract mp4 urls via mobile API (closes #12726) 2017-07-11 15:19:36 +08:00
708f6f511e [niconico] Fix authentication error handling (closes #12486) 2017-07-11 15:04:45 +08:00
bb13949197 [niconico] Check login errors (#12486) 2017-07-11 15:03:11 +08:00
c3c94ca4a4 [giantbomb] Extract m3u8 formats (closes #13626) 2017-07-10 21:34:27 +08:00
e3cd1fcdd1 [vlive:playlist] Relax and simplify 2017-07-10 04:32:24 +07:00
b71c18b434 [vlive:playlist] Add extractor 2017-07-10 04:24:04 +07:00
7bf539edcc [eagleplatform] Fix test 2017-07-10 00:14:41 +07:00
65c416dda8 release 2017.07.09 2017-07-09 20:16:38 +07:00
207acd8465 [ChangeLog] Actualize 2017-07-09 20:15:15 +07:00
71a1db8919 [dailymail] Add support for embeds 2017-07-09 20:06:24 +07:00
6e925598d6 [csjw] Add coding cookie 2017-07-09 19:18:12 +07:00
73cf76a93f [joj] Rewrite and add support for generic embeds (closes #13268) 2017-07-09 19:17:54 +07:00
256a746d21 [joj] Add extractor 2017-07-09 19:17:38 +07:00
58179eb7d9 [abc.net.au:iview] Extract more formats (closes #13492, closes #13489) 2017-07-09 17:55:40 +07:00
485cb37576 [egghead:course] Improve (closes #13370) 2017-07-09 17:30:49 +07:00
ed84454d35 [egghead:course] Fix extraction 2017-07-09 17:30:25 +07:00
a02682fd13 Keep in sync with ffmpeg's current malformed AAC bitstream wording (closes #13587) 2017-07-09 17:09:44 +07:00
0d2f0b0357 [csjw] Make description optional 2017-07-09 17:05:11 +07:00
c319d1c483 [csjw] Fix issues and improve extraction (closes #13525) 2017-07-09 17:01:05 +07:00
d2b9f362fa [cjsw] Add extractor 2017-07-09 17:01:00 +07:00
4328ddf82b [extractor/common] Add support for AMP tags in _parse_html5_media_entries 2017-07-09 16:29:52 +07:00
250b042c7e [generic] Add tests for #13557 2017-07-09 16:02:38 +07:00
665e945246 [eagleplatform] Add support for referrer protected videos (closes #13557) 2017-07-09 15:57:58 +07:00
5af2fd7fa0 [eagleplatform] Add support for another embed pattern (#13557) 2017-07-09 15:55:04 +07:00
15237fcd51 [veoh] Extend _VALID_URL 2017-07-09 14:54:52 +07:00
7a57730907 [npo:live] Fix live stream id extraction (closes #13568) 2017-07-09 14:21:40 +07:00
8b347a389e [googledrive] Fix height extraction (closes #13603) 2017-07-09 00:26:13 +07:00
a49804816c [dailymotion] Add support for new layout (close #13580) 2017-07-08 18:12:15 +07:00
eadd313321 [yam] Remove extractor
mymedia.yam.com is dead. An wikipedia user also pointed out that Yam's
blog service is no longer available. [1]

[1] https://zh.wikipedia.org/zh-tw/%E5%A4%A9%E7%A9%BA%E9%83%A8%E8%90%BD
2017-07-08 15:48:05 +08:00
d852c6bc59 [xhamster] Extract all formats and fix duration extraction (#13593) 2017-07-07 22:49:11 +07:00
00e5c36315 [xhamster] Add support for new URL schema (closes #13593) 2017-07-07 22:27:34 +07:00
8a04ade86b Credit @parmjitv for #13322, #13503, #13541, #13549 2017-07-06 23:15:23 +07:00
ab328411d5 Credit @orng for ruv (#13396) 2017-07-06 23:15:16 +07:00
ddeff4be3f Credit @gfabiano for #13382, #13385, #13415 2017-07-06 23:15:09 +07:00
60d4401c5e [espn] Extend _VALID_URL (fixes #13244) 2017-07-06 22:55:59 +07:00
dee2ff1d81 [test_utils] Fix tests under Windows 2017-07-06 00:25:37 +07:00
6554708252 [kaltura] Fix typo in subtitles extraction (closes #13569) 2017-07-05 23:20:50 +07:00
0a2e1b2e30 [vier] Adapt extraction to redesign (#13575) 2017-07-05 22:52:47 +07:00
babbc04d45 [xuite] Move to the new HTML5 API and reduce # of requests 2017-07-05 23:27:12 +08:00
609ff8ca19 [utils] Support attributes with no values in get_elements_by_attribute() 2017-07-05 23:27:12 +08:00
b6c9fe4162 release 2017.07.02 2017-07-02 20:17:10 +07:00
4d9ba27bba [ChangeLog] Actualize 2017-07-02 20:12:40 +07:00
50ae3f646e [thisoldhouse] Add more fallbacks for video id (closes #13541) 2017-07-02 20:06:15 +07:00
99a7e76240 [thisoldhouse] Update test 2017-07-02 20:05:11 +07:00
a3a6d01a96 [thisoldhouse] Fix video id extraction (closes #13540) 2017-07-02 20:04:51 +07:00
02d61a65e2 [xfileshare] Extend format regex (closes #13536) 2017-07-02 08:00:22 +07:00
9b35297be1 [extractors] Add import for tastytrade 2017-07-01 18:39:29 +07:00
4917478803 [ted] Fix extraction (closes #13535)) 2017-07-01 18:39:01 +07:00
54faac2235 [tastytrade] Add extractor (closes #13521) 2017-06-30 22:20:30 +07:00
c69701c6ab [extractor/common] Improve _json_ld 2017-06-30 22:19:06 +07:00
d4f8ce6e91 [dplayit] Relax video id regex (closes #13524) 2017-06-30 21:55:45 +07:00
b311b0ead2 [generic] Extract more generic metadata (closes #13527) 2017-06-30 21:42:04 +07:00
72d256c434 [bbccouk] Extend _VALID_URL 2017-06-29 22:29:28 +07:00
b2ed954fc6 [bbccouk] Capture and output error message (closes #13518) 2017-06-29 22:27:53 +07:00
a919ca0ad6 [cbsnews] Actualize test 2017-06-28 22:30:12 +07:00
88d6b7c2bd [cbsnews] Relax video info regex (fixes #13284) 2017-06-28 22:21:35 +07:00
fd1c5fba6b [facebook] Add test for plugin video embed (#13493) 2017-06-27 22:38:59 +07:00
0646e34c7d [facebook] Add support for plugin video embeds and multiple embeds (closes #13493) 2017-06-27 22:38:54 +07:00
bf2dc9cc6e [soundcloud] Fix tests 2017-06-27 21:26:46 +07:00
f1c051009b [soundcloud] Switch to https for API requests 2017-06-27 21:20:18 +07:00
33ffb645a6 [pandatv] Switch to https for API and download URLs 2017-06-26 22:11:09 +07:00
35544690e4 [pandatv] Add support for https URLs 2017-06-26 22:00:31 +07:00
136503e302 [ChangeLog] Update after #13494 2017-06-26 19:56:07 +08:00
4a87de72df [niconico] fix sp subdomain links 2017-06-25 21:30:05 +02:00
a7ce8f16c4 release 2017.06.25 2017-06-25 05:16:06 +07:00
a5aea53fc8 [ChangeLog] Actualize 2017-06-25 05:13:12 +07:00
0c7a631b61 [adobepass] Add support for ATTOTT MSO (DIRECTV NOW) (closes #13472) 2017-06-25 05:03:17 +07:00
fd9ee4de8c [wsj] Add support for barrons.com (closes #13470) 2017-06-25 02:15:35 +07:00
5744cf6c03 [ign] Add another video id pattern (closes #13328) 2017-06-25 01:59:15 +07:00
9c48b5a193 [raiplay:live] Improve and add test (closes #13414) 2017-06-25 01:49:27 +07:00
449c665776 [raiplay:live] Add extractor 2017-06-25 01:48:54 +07:00
23aec3d623 [redbulltv] Restore hls format prefix 2017-06-25 01:10:31 +07:00
27449ad894 [redbulltv] Add support for lives and segments (closes #13486)) 2017-06-25 01:09:12 +07:00
bd65f18153 [onetpl] Add support for videos embedded via pulsembed (closes #13482) 2017-06-24 18:33:31 +07:00
73af5cc817 [YoutubeDL] Skip malformed formats for better extraction robustness 2017-06-23 21:18:33 +07:00
b5f523ed62 [ooyala] Add test for missing stream['url']['data'] 2017-06-23 20:56:48 +07:00
4f4dd8d797 [ooyala] Make more robust 2017-06-23 20:56:21 +07:00
4cb18ab1b9 [ooyala] Skip empty format URLs (closes #13471, closes #13476) 2017-06-23 20:50:48 +07:00
ac7409eec5 [hgtv.com:show] Fix typo 2017-06-23 02:54:12 +07:00
170719414d release 2017.06.23 2017-06-23 02:13:21 +07:00
38dad4737f [ChangeLog] Actualize 2017-06-23 02:10:54 +07:00
ddbb4c5c3e [youtube] Adapt to new automatic captions rendition (closes #13467) 2017-06-23 02:00:19 +07:00
fa3ea7223a [hgtv.com:show] Relax video config regex and update test (closes #13279, closes #13461) 2017-06-23 00:42:42 +07:00
0f4a5a73e7 [drtuber] Fix formats extraction (fixes 12058) 2017-06-23 00:08:36 +07:00
18166bb8e8 [youporn] Fix upload date extraction 2017-06-22 00:47:02 +07:00
d4893e764b [youporn] Improve formats extraction 2017-06-22 00:40:15 +07:00
97b6e30113 [youporn] Fix title extraction (closes #13456) 2017-06-22 00:20:45 +07:00
9be9ec5980 [googledrive] Fix formats' sorting (closes #13443) 2017-06-20 22:58:33 +07:00
048b55804d [watchindianporn] Fix extraction (closes #13411) 2017-06-20 04:30:45 +07:00
6ce79d7ac0 [abcotvs] Fix test md5 2017-06-20 04:07:00 +07:00
1641ca402d [vimeo] Add fallback mp4 extension for original format 2017-06-20 01:27:59 +07:00
85cbcede5b [ruv] Improve, extract all formats and metadata (closes #13396) 2017-06-19 23:46:03 +07:00
Orn
a1de83e5f0 [ruv] Add extractor 2017-06-19 23:45:45 +07:00
fee00b3884 [viu] Fix extraction on older python 2.6 2017-06-19 22:57:37 +07:00
2d2132ac6e [adobepass] Fix extraction on older python 2.6 2017-06-19 22:54:53 +07:00
cc2ffe5afe [pandora.tv] Fix upload_date extraction (closes #12846) 2017-06-19 16:20:36 +08:00
560050669b [asiancrush] Add extractor (closes #13420) 2017-06-18 20:18:51 +07:00
eaa006d1bd release 2017.06.18 2017-06-18 00:16:49 +07:00
a6f29820c6 [ChangeLog] Actualize 2017-06-18 00:15:43 +07:00
1433734c35 [downloader/common] Use utils.shell_quote for debug command line 2017-06-17 23:50:21 +07:00
aefce8e6dc [utils] Use compat_shlex_quote in shell_quote 2017-06-17 23:48:58 +07:00
8b6ac49ecc [postprocessor/execafterdownload] Encode command line (closes #13407) 2017-06-17 23:16:53 +07:00
b08e235f09 [compat] Fix compat_shlex_quote on Windows (closes #5889, closes #10254) 2017-06-17 23:14:24 +07:00
be80986ed9 [postprocessor/metadatafromtitle] Fix missing optional meta fields (closes #13408) 2017-06-17 19:05:10 +07:00
473e87064b [devscripts/prepare_manpage] Fix deprecated escape sequence on py36 2017-06-17 17:37:25 +08:00
4f90d2aeac [Makefile] Excluding __pycache__ correctly (#13400) 2017-06-17 17:09:24 +08:00
b230fefc3c [polskieradio] Fix extraction 2017-06-16 04:57:56 +07:00
96a2daa1ee [extractor/common] Improve jwplayer subtitles extraction 2017-06-15 23:40:39 +07:00
0ea6efbb7a [xfileshare] Add support for fastvideo.me 2017-06-15 21:41:49 +07:00
6a9cb29509 [extractor/common] Fix json dumping with --geo-bypass
The line "[debug] Using fake IP %s (%s) as X-Forwarded-For." was printed
to stdout even with -j/-J, which breaks the resultant JSON.
2017-06-15 13:04:36 +08:00
ca27037171 [bilibili] Fix extraction of videos with double quotes in titles
Closes #13387
2017-06-15 11:19:03 +08:00
0bf4b71b75 [4tube] Fix extraction (closes #13381) 2017-06-15 04:16:50 +07:00
5215f45327 [disney] Add support for disneychannel.de 2017-06-15 04:13:04 +07:00
0a268c6e11 [extractor/common] Improve jwplayer formats extraction (closes #13379) 2017-06-14 22:02:15 +07:00
7dd5415cd0 [npo] Improve _VALID_URL (closes #13376) 2017-06-14 21:33:40 +07:00
b5dc33daa9 [corus] Add support for showcase.ca 2017-06-13 23:27:27 +07:00
97fa1f8dc4 [corus] Add support for history.ca (closes #13359) 2017-06-13 23:16:21 +07:00
b081f53b08 [compat] Add compat_HTMLParseError to __all__ 2017-06-12 02:36:43 +07:00
cb1e6d8985 release 2017.06.12 2017-06-12 02:23:17 +07:00
9932ac5c58 [ChangeLog] Actualize 2017-06-12 02:01:15 +07:00
bf87c36c93 [xfileshare] PEP 8 2017-06-12 02:01:12 +07:00
b4a3d461e4 [utils] Handle HTMLParseError in extract_attributes (closes #13349) 2017-06-12 01:52:24 +07:00
72b409559c [compat] Introduce compat_HTMLParseError 2017-06-12 01:50:32 +07:00
534863e057 [xfileshare] Add support for rapidvideo (closes #13348) 2017-06-12 00:16:47 +07:00
16bc958287 [xfileshare] Modernize and pass referrer 2017-06-12 00:14:04 +07:00
624bd0104c [rutv] Add support for testplayer.vgtrk.com (closes #13347) 2017-06-11 21:36:19 +07:00
28a4d6cce8 [newgrounds] Extract more metadata (closes #13232) 2017-06-11 21:30:06 +07:00
2ae2ffda5e [utils] Improve unified_timestamp 2017-06-11 21:27:22 +07:00
70e7967202 [newgrounds:playlist] Add extractor (closes #10611) 2017-06-11 20:50:55 +07:00
6e999fbc12 [newgrounds] Improve formats and uploader extraction (closes #13346) 2017-06-11 19:44:44 +07:00
7409af9eb3 [msn] Fix formats extraction 2017-06-11 08:56:53 +07:00
4e3637034c [extractor/generic] Ensure format id is unicode string 2017-06-10 23:56:20 +07:00
1afd0b0da7 [extractor/common] Return unicode string from _match_id 2017-06-09 00:40:03 +07:00
7515830422 [turbo] Ensure format id is string 2017-06-09 00:31:56 +07:00
f5521ea209 [sexu] Ensure height is int 2017-06-09 00:30:23 +07:00
34646967ba [jove] Ensure comment count is int 2017-06-09 00:29:20 +07:00
e4d2e76d8e [golem] Ensure format id is string 2017-06-09 00:27:11 +07:00
87f5646937 [gfycat] Ensure filesize is int 2017-06-09 00:24:23 +07:00
cc69a3de1b [foxgay] Ensure height is int 2017-06-09 00:22:14 +07:00
15aeeb1188 [flickr] Ensure format id is string 2017-06-09 00:20:07 +07:00
1693bebe4d [sohu] Fix numeric fields 2017-06-09 00:16:42 +07:00
4244a13a1d [safari] Improve authentication detection (closes #13319) 2017-06-08 23:20:48 +07:00
931adf8cc1 [liveleak] Ensure height is int (closes #13313) 2017-06-08 22:54:30 +07:00
c996943418 [YoutubeDL] Sanitize more fields (#13313) 2017-06-08 22:53:14 +07:00
76e6378358 [README.md] Improve man page formatting 2017-06-08 22:02:42 +07:00
a355b57f58 [README.md] Clarify output template references (closes #13316) 2017-06-08 21:52:19 +07:00
1508da30c2 [streamango] Skip download for test (closes #13292) 2017-06-07 21:53:40 +07:00
eb703e5380 [streamango] Make title optional 2017-06-07 21:53:33 +07:00
0a3924e746 [rtlnl] Improve _VALID_URL (closes #13295) 2017-06-06 21:21:44 +07:00
e1db730d86 [tvplayer] Fix extraction (closes #13291) 2017-06-06 00:13:57 +07:00
537191826f release 2017.06.05 2017-06-05 00:48:07 +07:00
130880ba48 [ChangeLog] Actualize 2017-06-05 00:43:38 +07:00
f8ba3fda4d Credit @jktjkt for dvtv formats (#13063) 2017-06-05 00:38:44 +07:00
e1b90cc3db Credit @mikf for beam:vod (#13032) 2017-06-05 00:35:41 +07:00
43e6579558 Credit @adamvoss for bandcamp:weekly (#12758) 2017-06-04 23:22:19 +07:00
6d923aab35 [bandcamp:weekly] Improve and extract more metadata (closes #12758) 2017-06-04 23:21:30 +07:00
62bafabc09 [bandcamp:weekly] Add extractor 2017-06-04 23:21:07 +07:00
9edcdac90c [pornhub:uservideos] Add missing raise 2017-06-04 20:39:55 +07:00
cd138d8bd4 [pornhub:playlist] Fix extraction (closes #13281) 2017-06-04 15:54:19 +07:00
cd750b731c [godtv] Remove extractor (closes #13175) 2017-06-03 22:08:12 +07:00
4bede0d8f5 [YoutubeDL] Don't emit ANSI escape codes on Windows 2017-06-03 19:14:23 +07:00
f129c3f349 [safari] Fix typo (closes #13252) 2017-06-02 01:03:51 +07:00
39d4c1be4d [youtube] Improve chapters extraction (closes #13247) 2017-06-01 23:29:45 +07:00
f7a747ce59 [1tv] Lower preference for http formats (closes #13246) 2017-06-01 22:19:52 +07:00
4489d41816 [francetv] Relax _VALID_URL 2017-06-01 00:15:15 +07:00
87b5184a0d [drbonanza] Fix extraction (closes #13231) 2017-05-31 23:56:32 +07:00
c56ad5c975 [packtpub] Fix authentication(closes #13240) 2017-05-31 15:44:29 +01:00
6b7ce85cdc [README.md] Mention http_dash_segments protocol 2017-05-30 23:50:48 +07:00
d10d0e3cf8 [README.md] Add an example for how to use .netrc on Windows
That's a Python bug: http://bugs.python.org/issue28334
Most likely it will be fixed in Python 3.7: https://github.com/python/cpython/pull/123
2017-05-29 14:58:07 +08:00
941ea38ef5 release 2017.05.29 2017-05-29 00:42:18 +07:00
99bea8d298 [ChangeLog] Actualize 2017-05-29 00:33:56 +07:00
a49eccdfa7 [youtube] Parse player_url if format URLs are encrypted or DASH MPDs are requested
Fixes #13211
2017-05-28 20:20:20 +08:00
a846173d93 [xhamster] Simplify (closes #13216) 2017-05-28 07:55:56 +07:00
78e210dea5 [xhamster] Fix author and like/dislike count extraction 2017-05-28 07:55:07 +07:00
8555204274 [xhamster] Extract categories (closes #11728) 2017-05-28 07:50:15 +07:00
164fcbfeb7 [abcnews] Improve and remove duplicate test (closes #12851) 2017-05-28 07:06:56 +07:00
bc22df29c4 [abcnews] Add support for embed URLs 2017-05-28 07:06:29 +07:00
7e688d2f6a [gaskrank] Improve (closes #12493) 2017-05-28 06:47:38 +07:00
5a6d1da442 [gaskrank] Fix extraction 2017-05-28 06:47:30 +07:00
703751add4 [medialaan] PEP 8 (closes #12774) 2017-05-28 06:27:57 +07:00
4050be78e5 [medialaan] Fix videos with missing videoUrl
A rough trick to get around the two different json styles medialaan seems to be using.
Fix for these example videos:
https://vtmkzoom.be/video?aid=45724
https://vtmkzoom.be/video?aid=45425
2017-05-28 06:27:52 +07:00
4d9fc40100 [dvtv] Improve and fix playlists support (closes #13063) 2017-05-28 06:19:54 +07:00
765522345f [dvtv] Parse adaptive formats as well
The old code hit an error when it attempted to parse the string
"adaptive" for video height. Actually parsing the returned playlists is
a good idea because it adds more output formats, including some
audio-only-ones.
2017-05-28 06:19:46 +07:00
6bceb36b99 [beam] Improve and add support for mixer.com (closes #13032) 2017-05-28 05:43:04 +07:00
1e0d65f0bd [beam:vod] Add extractor 2017-05-28 05:42:23 +07:00
03327bc9a6 [cbsinteractive] Relax _VALID_URL (closes #13213) 2017-05-27 22:37:24 +07:00
b407d8533d [utils] Drop an compatibility wrapper for Python < 2.6
addinfourl.getcode is added since Python 2.6a1. As youtube-dl now
requires 2.6+, this is no longer necessary.

See 9b0d46db11
2017-05-27 23:05:02 +08:00
20e2c9de04 [adn] fix formats extraction 2017-05-26 20:00:44 +01:00
d16c0121b9 [youku] Extract more metadata (closes #10433) 2017-05-27 00:08:37 +08:00
7f4c3a7439 [cbsnews] Fix extraction (closes #13205) 2017-05-26 22:42:27 +07:00
28dbde9cc3 release 2017.05.26 2017-05-26 22:29:16 +07:00
cc304ce588 [ChangeLog] Actualize 2017-05-26 22:27:56 +07:00
98a0618941 [ChangeLog] Update after the fix for #11381 2017-05-26 23:22:54 +08:00
fd545fc6d1 Revert "[youtube] Don't use the DASH manifest from 'get_video_info' if 'use_cipher_signature' is True (#5118)"
This reverts commit 87dc451108.
2017-05-26 23:22:54 +08:00
97067db2ae [bbc] Add support for authentication 2017-05-26 22:12:24 +07:00
c130f0a37b [tudou] Merge into youku extractor (fixes #12214)
Also, there are no tudou playlists anymore. All playlist URLs points to youku
playlists.
2017-05-26 23:04:42 +08:00
d3d4ba7f24 [youku:show] Fix extraction 2017-05-26 21:59:16 +08:00
5552c9eb0f [utils] Recognize more patterns in strip_jsonp()
Used in Youku Show pages
2017-05-26 21:58:18 +08:00
59ed87cbd9 [youku] Fix extraction (closes #13191) 2017-05-26 19:16:52 +08:00
b7f8749304 [udemy] Fix extraction for outputs' format entries without URL (closes #13192) 2017-05-25 22:28:26 +07:00
5192ee17e7 [postprocessor/ffmpeg] Fix metadata filename handling on Python 2
Fixes #13182
2017-05-25 22:07:03 +08:00
e834f04400 [vimeo] Fix formats' sorting (closes #13189) 2017-05-24 22:58:16 +07:00
884d09f330 [cbsnews] fix extraction for 60 Minutes videos 2017-05-24 09:44:41 +01:00
9e35298f97 Merge pull request #12861 from Tithen-Firion/cbsinteractive-fix
[cbsinteractive] update extractor and test cases
2017-05-24 10:21:33 +02:00
0551f1b07b Credit @gritstub for vevo fix (#12879) 2017-05-23 21:47:40 +07:00
de53511201 Credit @timendum for rai (#11790) and mediaset (#12964) 2017-05-23 00:41:53 +07:00
2570e85167 release 2017.05.23 2017-05-23 00:17:48 +07:00
9dc5ab041f [ChangeLog] Actualize 2017-05-23 00:15:44 +07:00
01f3c8e290 Credit @fredbourni for noovo (#12792) 2017-05-23 00:08:14 +07:00
06c1b3ce07 Credit @mphe for streamango (#12643) 2017-05-23 00:07:31 +07:00
0b75e42dfb Credit @zurfyx for atresplayer improvements (#12548) 2017-05-23 00:00:49 +07:00
a609e61a90 [downloader/external] Pass -loglevel to ffmpeg downloader (closes #13183) 2017-05-22 23:40:07 +07:00
afdb387cd8 [streamcz] Add support for subtitles 2017-05-21 15:41:52 +07:00
dc4e4f90a2 [youtube] Modernize 2017-05-21 01:18:56 +07:00
fdc20f87a6 [youtube] Fix DASH manifest signature decryption (closes #8944) 2017-05-21 01:11:37 +07:00
35a2d221a3 [toggle] Relax _VALID_URL (closes #13172) 2017-05-20 23:06:30 +07:00
daa4e9ff90 [adobepass] Add support for Brighthouse MSO 2017-05-20 20:50:46 +07:00
2ca29f1aaf [toypics] Improve and modernize 2017-05-20 01:29:33 +07:00
77d682da9d [toypics] Fix extraction 2017-05-20 01:18:03 +07:00
8fffac6927 [njpwworld] Fix extraction (closes #13162) 2017-05-19 23:11:02 +07:00
5f6fbcea08 [hitbox] Add support for smashcast.tv (closes #13154) 2017-05-19 22:34:00 +07:00
00cb0faca8 [mitele] Update app key regex 2017-05-19 21:54:57 +07:00
bfdf6fcc66 release 2017.05.18.1 2017-05-18 23:00:03 +07:00
bcaa1dd060 [ChangeLog] Actualize 2017-05-18 22:58:14 +07:00
0e2d626ddd [jsinterp] Fix typo and cleanup regexes (closes #13134) 2017-05-18 22:57:38 +07:00
9221d5d7a8 [ChangeLog] Fix typo 2017-05-18 22:37:32 +07:00
9d63e57d1f release 2017.05.18 2017-05-18 22:30:37 +07:00
3bc1eea0d8 [ChangeLog] Actualize 2017-05-18 22:29:25 +07:00
7769f83701 [jsinterp] Add support for quoted names and indexers (closes #13123, closes #13130) 2017-05-18 22:18:33 +07:00
650bd94716 [vier] Relax regexes and extract more metadata (closes #12539) 2017-05-17 23:39:01 +07:00
36b226d48f [vier] Extract more info
Extract the `episode_number` and `upload_date`. Also extract the real
`description`.
2017-05-17 23:38:50 +07:00
f2e2f0c777 [extractor/common] Fix rtmp and rtsp formats' URLs in _extract_wowza_formats 2017-05-17 22:20:25 +07:00
6f76679804 [extractor/common] Add support for schemeless URLs in _extract_wowza_formats (closes #13088, closes #13092) 2017-05-16 22:11:34 +07:00
7073015a23 [vier] PEP 8 and cleanup 2017-05-15 22:00:53 +07:00
89fd03079b [vier] Improve extraction
+ Add support for authentication
* Bypass authentication when no credentials provded
* Improve extraction robustness
2017-05-15 21:46:55 +07:00
1c45b7a8a9 [dailymail] Fix sources extraction (closes #13057) 2017-05-14 12:47:19 +07:00
60f5c9fb19 [utils] Recognize more audio codecs (#13081) 2017-05-14 12:33:33 +07:00
c360e641e9 [dailymotion] Extend _VALID_URL (closes #13079) 2017-05-14 09:55:40 +07:00
6f3c632c24 release 2017.05.14 2017-05-14 07:38:40 +07:00
09b866e171 [ChangeLog] Actualize 2017-05-14 07:37:31 +07:00
166d12b00c [options] PEP 8 2017-05-14 07:33:24 +07:00
2b8e6a68f8 [extractor/generic] Add test for mediaset embed 2017-05-14 06:40:19 +07:00
d105a7edc6 [mediaset] Fix upload date 2017-05-14 06:39:47 +07:00
5d29af3d15 [extractor/generic] Add support for mediaset embeds 2017-05-14 06:29:16 +07:00
ca04de463d [mediaset] Add support for shortcut 2017-05-14 06:28:40 +07:00
946826eec7 [extractor/generic] Remove duplicate limelight code 2017-05-14 06:17:34 +07:00
76d5a36391 [extractor/common] Respect Width and Height attributes in ISM manifests 2017-05-14 06:11:45 +07:00
56f9c77f0e [mediaset] Improve extraction (closes #12708, closes #12964) 2017-05-14 05:30:13 +07:00
0de136341a [mediaset] Add extractor 2017-05-14 05:30:02 +07:00
1339ecb2f8 [orf:radio] Cleanup _VALID_URLs (closes #11643) 2017-05-14 04:31:20 +07:00
efe9316703 [orf:radio] Fix extraction
Since oe1.orf.at has been updated, both ORF radios supported by youtube_dl
use the same API. This commit honors this fact by merging both extractors
into one.
2017-05-14 04:31:14 +07:00
851a01aed6 [aljazeera] Extend _VALID_URL 2017-05-14 00:57:02 +07:00
b845766597 [imdb] Relax _VALID_URL (closes #13056) 2017-05-14 00:32:50 +07:00
fa26734e07 [postprocessor/metadatafromtitle] Add support regex syntax for --metadata-from-title (closes #13065) 2017-05-14 00:03:15 +07:00
12f01118b0 [francetv] Add support for mobile.france.tv (closes #13068) 2017-05-13 21:57:00 +07:00
7fc60f4ee9 [upskill] Add extractor (closes #13043) 2017-05-13 21:52:59 +07:00
58bb440283 [extractor/generic] Extract wistia embed code into separate method 2017-05-13 21:51:58 +07:00
7ad4362357 [thescene] fix extraction(closes #13061) 2017-05-12 16:37:09 +01:00
6c52477f59 [condenast] improve embed support 2017-05-12 16:37:09 +01:00
116283ff64 [liveleak] Fix extraction (#12053) 2017-05-12 19:15:33 +08:00
7274f3d0e9 [douyu] Support Douyu shows (closes #12228) 2017-05-12 18:44:24 +08:00
3166b1f0ac [myspace] Improve _VALID_URL (closes #13040) 2017-05-10 22:35:46 +07:00
39ee263819 use platform=desktop in assets url(closes #13041) 2017-05-10 08:50:30 +01:00
a7ed6b341c release 2017.05.09 2017-05-09 04:20:13 +07:00
cbd84b5817 [ChangeLog] Actualize 2017-05-09 23:17:22 +07:00
6d1ded7502 [francetv] Adapt to site redesign (closes #13034) 2017-05-09 23:07:01 +07:00
5d0968f0af [packtpub] add support for authentication(closes #12622) 2017-05-09 11:15:14 +01:00
8d65880e24 [drtv] Improve extraction and update tests (closes #13013, closes #13016) 2017-05-09 15:37:09 +07:00
b972fb037b [drtv] Lower preference for SignLanguage formats (closes #13013) 2017-05-09 15:36:02 +07:00
5996d21aea [cspan] add support for brightcove live embeds(closes #13028) 2017-05-09 00:51:12 +01:00
afa0200bf0 [vrv] extract dash formats and subtitles 2017-05-08 20:04:40 +01:00
e9137224b3 [YoutubeDL] Force restrict filenames when no locale is set for python 2 as well (#13027) 2017-05-09 01:14:02 +07:00
804181dda9 [funimation] remove codes related to old login method and update test 2017-05-08 18:58:13 +01:00
8fa17117df [funimation] fix authentication(closes #13021) 2017-05-08 18:13:58 +01:00
3b859145c2 [adultswim] Fix Extraction(closes #8640)(closes #10950)(closes closes #11042)(closes #12121)
- add support for adobe pass authentication
- add support for live streams
- add support for show pages
2017-05-08 15:07:40 +01:00
04c09f1961 [turner] extract thumbnail and is_live and strip description 2017-05-08 15:07:40 +01:00
bf82b87323 [nonktube] Use econfig nuevo URL 2017-05-08 20:13:22 +07:00
b6eb74e340 [nonktube] Add extractor (closes #8647, closes #13024) 2017-05-08 20:10:39 +07:00
3d40084b83 [nuevo] Pass headers to _extract_nuevo 2017-05-08 20:03:38 +07:00
52294cdda7 [nbc] remove unused imports and extract permalink from modified urls 2017-05-07 09:31:14 +01:00
2eeb588efe [nbc] improve extraction(closes #12364) 2017-05-07 08:59:31 +01:00
4ac0f573ef release 2017.05.07 2017-05-07 04:51:34 +07:00
3892a9f4ab [ChangeLog] Actualize 2017-05-07 04:44:54 +07:00
3995d37da5 [youtube] Fix TFA (#12927) 2017-05-07 04:19:11 +07:00
e4a75d7932 [test_youtube_chapters] PEP 8 2017-05-07 00:00:11 +07:00
e00eb564e9 [youtube] Fix authentication (closes #12927) 2017-05-06 23:58:47 +07:00
10c87c151b [utils] Rename try_multipart_encode to _multipart_encode_impl
To state that this is an internal function and people should be careful
when using it outside youtube-dl.
2017-05-06 19:06:18 +08:00
228cd9bb90 [bilibili] Fix video downloading (closes #13001) 2017-05-06 18:58:38 +08:00
566fbbaefd [rmcdecouverte] Improve (closes #12937) 2017-05-06 17:56:10 +07:00
74c09c852a [rmcdecouverte] Fix extraction 2017-05-06 17:56:10 +07:00
fd178b8748 [theplatform] extract chapters 2017-05-06 07:19:07 +01:00
a57a8e9918 [test_youtube_chapters] Add coding cookie 2017-05-06 05:30:56 +07:00
1f9fefe7f5 [crackle] Update test 2017-05-06 03:39:14 +07:00
8b4774dcac [bandcamp] Fix thumbnail extraction 2017-05-06 03:35:42 +07:00
a99cc4ca16 [pornhub] Extend _VALID_URL (closes #12996) 2017-05-06 02:46:37 +07:00
9cafc3fd8b [youtube] Extract chapters 2017-05-06 02:27:06 +07:00
329e3dd5ad [nrk] Extract chapters 2017-05-05 22:59:15 +07:00
1d9e0a4f40 [vice] update tests and add support for ooyala embeds in article pages 2017-05-05 16:13:12 +01:00
7ad53cb7ff [laola1tv] PEP 8 2017-05-05 21:59:23 +07:00
b2ad479d17 [utils] Fix multipart_encode for Python < 3.5 2017-05-05 20:51:59 +08:00
4ac6dc3732 [vice] Support Vice articles (closes #12968) 2017-05-05 20:26:51 +08:00
cc7bda4fff [vice] Fix extraction for non en_us videos (closes #12967) 2017-05-05 20:01:02 +08:00
50ad078b7b [gdcvault] Fix extraction for videos with gdc-player.html
Closes #12733
2017-05-05 15:13:40 +08:00
4947f13cd0 [pbs] Improve multipart video support (closes #12981) 2017-05-04 22:42:49 +07:00
7f09e523e8 [laola1tv:embed] Fix tests 2017-05-04 22:41:47 +07:00
4fe14732a2 [laola1tv] fix extraction(closes #12880) 2017-05-04 16:07:08 +01:00
ff6f9a6704 [extractor/common] fix typo in _extract_akamai_formats 2017-05-04 16:07:08 +01:00
c89267d31a Merge branch 'master' into openload-phantomjs-method 2017-05-04 11:00:06 +02:00
0c26548601 [cda] Implement birthday verification (closes #12789) 2017-05-04 16:26:17 +08:00
5401bea27f [leeco] Fix extraction (closes #12974)
Seems on mobile devices a similar API is used, but I always get an AD
with mimicking that API.
2017-05-04 03:18:56 +08:00
7a6d33a9a5 [pbs] extract chapters information 2017-05-02 20:41:48 +01:00
fa2a36d9bc [ffmpeg] add support for chapters field postprocessing 2017-05-02 20:41:48 +01:00
55949fede6 [common] introduce chapters field 2017-05-02 20:41:48 +01:00
7fc875195f [amp] imporove thumbnail and subtitle extraction 2017-05-02 00:06:58 +01:00
c6fe5a7e12 [douyutv] Update test 2017-05-02 03:45:27 +08:00
ae21d2fd94 [dotsub] Update test 2017-05-02 02:56:44 +08:00
77481f1386 [democracynow] Update test 2017-05-02 01:38:31 +07:00
d86d169dd5 [dailymotion] Add working test 2017-05-02 01:37:23 +07:00
b9f9f361fa [crunchyroll] Update test 2017-05-02 00:56:51 +07:00
ab39a25c75 [foxsports] fix extraction(closes #12945) 2017-05-01 09:02:41 +01:00
a146fa1c68 [coub] Update test and remove comment count extraction 2017-05-01 05:54:44 +07:00
e0c1e9a98c release 2017.05.01 2017-05-01 01:39:52 +07:00
086041e2f8 [ChangeLog] Actualize 2017-05-01 01:34:51 +07:00
74da856544 [infoq] Make audio format extraction non fatal (closes #12938) 2017-05-01 01:23:05 +07:00
9edf47df7b [brightcove] Allow whitespace around attribute names in embedded code 2017-05-01 01:03:47 +07:00
238cec17ae [extractor/anvato] PEP 8 2017-04-30 22:04:21 +07:00
50534b7158 [downloader/fragment] PEP 8 2017-04-30 22:04:01 +07:00
9cd4209724 [zaq1] Improve extraction (closes #12693) 2017-04-30 21:46:05 +07:00
33a81c2c6f [extractor/common] Extract view count from JSON-LD 2017-04-30 21:45:59 +07:00
deef31955b [utils] Improve unified_timestamp
Seen at http://zaq1.pl/video/xev0e
2017-04-30 21:45:53 +07:00
9dac2cec2d [zaq1] Add new extractor 2017-04-30 21:45:47 +07:00
6ec371cd9e [xvideos] Extract og:duration (closes #12828) 2017-04-30 18:14:01 +07:00
13081db1f5 [xvideos] Add video duration 2017-04-30 18:10:49 +07:00
b07ea5eaec [vevo] Modernize 2017-04-30 17:58:22 +07:00
5599253009 [vevo] Fix extraction (config.token.key) 2017-04-30 17:56:10 +07:00
98ce1a3fd3 [utils] add video/mp2t to mimetype2ext 2017-04-30 09:03:10 +01:00
5ff1bc0cc1 [YoutubeDL] write raw subtitle files 2017-04-29 20:03:03 +01:00
ba5c3caf88 [washingtonpost] Fix invalid escape sequence on Python 3.6 2017-04-30 02:15:28 +08:00
b5c39537be [noovo] Improve extraction (closes #12792) 2017-04-30 00:24:25 +07:00
1c7c76e4fb [noovo] Add extractor 2017-04-30 00:24:19 +07:00
557194591a [washingtonpost] Add support for embeds (closes #12699) 2017-04-29 23:07:26 +07:00
27e70a8f6c Merge pull request #12869 from Tithen-Firion/cbc-update-tests
[cbc] update test cases
2017-04-29 21:34:18 +08:00
a4c81e4968 [yandexmusic:playlist] Fix extraction for python 3 (closes #12888) 2017-04-29 20:23:26 +07:00
7986c3abcd [anvato] Improve extraction (closes #12913)
* Promote to regular shortcut based extractor
* Add mcp to access key mapping table
* Add support for embeds extraction
* Add support for anvato embeds in generic extractor
2017-04-29 19:49:04 +07:00
a1ebfd4494 Merge pull request #12854 from Tithen-Firion/appletrailer-test-fix
[appletrailers] update test cases
2017-04-29 19:24:38 +08:00
7552f96352 [openload] Add required version 2017-04-29 12:41:57 +02:00
98f9d87381 [phantomjs] Add required version checking 2017-04-29 12:41:42 +02:00
fcace2d1ad [openload] raise not found before executing js 2017-04-29 10:30:45 +02:00
d19093bd50 Merge pull request #12906 from Tithen-Firion/clean-html-fix
[utils] Fix inconsistent output of clean_html
2017-04-29 15:58:45 +08:00
24eb7c2578 [xtube] Fix extraction with non-standard JSON 'sources'
Closes #12734

Thanks @paulguy for the fix!
2017-04-29 15:55:08 +08:00
e7db6759e4 [downloader/external] Properly handle live stream downloading cancellation (closes #8932) 2017-04-29 04:33:35 +07:00
b364c87c42 [tvplayer] Fix extraction (closes #12908) 2017-04-29 03:46:08 +07:00
9222d94510 [test_utils] Add one more clean_html test 2017-04-28 18:05:14 +02:00
edd9221cd2 [utils] Fix inconsistent output of clean_html
`\s` in Python 2.x doesn't match unicode whitespace characters by
default
2017-04-28 17:34:27 +02:00
bc8a2ea071 release 2017.04.28 2017-04-28 18:30:03 +07:00
7527923371 [ChangeLog] Actualize 2017-04-28 18:27:29 +07:00
20783b8b50 [aenetworks] fix extraction for shows with single season 2017-04-28 12:04:56 +01:00
bf2a5555c0 [go] add support for Disney, DisneyJunior and DisneyXD show pages 2017-04-28 09:48:52 +01:00
fb8e8b2d16 [adobepass] use geo verification headers for all requests 2017-04-28 09:48:52 +01:00
b62985a9a5 [youtube] Recognize another HTML5 player URL (#12885) 2017-04-28 16:25:04 +08:00
e31fed95b4 [youtube] Recognize new locale-based player URLs (fixes #12885) 2017-04-28 15:48:30 +08:00
3fd0f70f6a [cbslocal] Update test 2017-04-28 04:26:59 +07:00
33c62efc32 [collegerama] Update tests 2017-04-28 04:00:49 +07:00
6b4ddd336c [afreecatv] Fix title extraction 2017-04-28 04:00:15 +07:00
c12b4b80f8 [archiveorg] Update test 2017-04-28 03:48:32 +07:00
064fafe932 [appleconnect] Update test 2017-04-28 03:47:25 +07:00
ac1a5b9a12 [audioboom] Update test 2017-04-28 03:36:28 +07:00
a15777491a [atresplayer] Update test 2017-04-28 03:32:25 +07:00
d8571dd6bf [bleacherreport] Update tests 2017-04-28 03:28:26 +07:00
c0fa4245ce [downloader/fragment] Remove assert for resume_len when no fragments downloaded
This may be incorrect due some header (e.g. flv header in f4m downloader)
2017-04-28 03:26:19 +07:00
8814ae42bc [beeg] Update test 2017-04-28 03:14:11 +07:00
0f63dc2402 [bandcamp] Update test 2017-04-28 03:13:12 +07:00
dde97ea8da [canalc2] Update test 2017-04-28 03:07:42 +07:00
30bb6ce1a4 [test_InfoExtractor] Fix test_parse_m3u8_formats 2017-04-28 03:01:43 +07:00
c89b49f743 [extractor/common] Add manifest_url for explicit group rendition formats 2017-04-28 03:00:14 +07:00
6f4a888416 [br] Update test 2017-04-28 02:53:11 +07:00
f5edd7ae51 [clipfish] Update test 2017-04-28 02:51:30 +07:00
96820c1c6b [cbsinteractive] extract formats with CBSIE 2017-04-27 20:23:52 +02:00
c95e2b5911 [cbc] update test cases 2017-04-27 18:07:07 +02:00
374560f018 [test_download] Fix order when testing file's md5 2017-04-27 22:27:34 +07:00
ff99fe529e Don't list master m3u8 playlists in format list (closes #12832) 2017-04-27 21:53:17 +07:00
e095109da1 [cbsinteractive] update test cases 2017-04-27 15:40:17 +02:00
d68afc5bc9 [cbsinteractive] fix extractor 2017-04-27 15:27:01 +02:00
76c1951036 [appletrailers] update test cases 2017-04-27 10:04:21 +02:00
e8bfe2a946 [streamable] Add support for new embedded URL schema 2017-04-26 23:39:53 +07:00
3dc8b61b7f [arte:+7] Relax _VALID_URL (closes #12837) 2017-04-26 01:55:29 +07:00
a82f41841d release 2017.04.26 2017-04-26 00:06:12 +07:00
30a4ab191a [ChangeLog] Actualize 2017-04-26 00:03:13 +07:00
ac9c69ace7 [extractor/common] Improve jwplayer regex 2017-04-25 23:46:05 +07:00
85f6de25e4 [downloader/fragment] Clarify current_fragment's index and mark as experimental 2017-04-25 23:33:35 +07:00
538eee7b6a Add missing test m3u8 file 2017-04-25 22:26:30 +07:00
9f54ae2873 Ignore and clean *.ytdl files 2017-04-25 22:42:55 +08:00
01cb57016f [iqiyi] Fix extraction of Yule videos 2017-04-25 22:23:57 +08:00
40e41780f1 [phantomjs] add cookie support 2017-04-25 15:12:54 +02:00
da57ebaf84 [openload] separate PhantomJS code from extractor 2017-04-25 01:06:14 +02:00
290f64dbaa [downloader/fragment] Improve .ytdl format and start documenting 2017-04-24 23:50:20 +07:00
adb4b03cd5 [downloader/fragment] Don't process ytdl file when it's not needed yet 2017-04-24 23:05:56 +07:00
0eee52f34b Introduce --keep-fragments 2017-04-24 03:09:08 +07:00
d3f0687cf7 [downloader/fragment] Use temp file for current fragment 2017-04-24 02:54:17 +07:00
a4d6cf970c [YoutubeDL] Fix output template for missing timestamp (closes #12796) 2017-04-24 00:50:39 +07:00
3019cb0c99 [extractor/common] Rephrase comment 2017-04-23 11:52:07 +07:00
ddd258f922 [test_InfoExtractor] Add m3u8 parsing test for NAME attribute in EXT-X-STREAM-INF tag 2017-04-23 11:49:57 +07:00
07ad0cf34f [vidio] Improve and sort formats 2017-04-23 11:48:51 +07:00
9c99bef704 [extractor/common] Use float for scaled tbr 2017-04-23 11:33:49 +07:00
ffbc8386b9 [brightcove] match only video elements with data-video-id attribute 2017-04-22 22:26:20 +01:00
4abdba643c [downloader/fragment] remove unused code 2017-04-22 18:19:47 +01:00
3e0304fe6e [downloader/fragment] use the documented names for fragment progress_hooks fields 2017-04-22 16:42:24 +01:00
fbf56be213 [iqiyi] Fix playlist detection (#12504) 2017-04-22 22:11:37 +08:00
54f54fcca7 [socks] Report errors elegantly when credentails are required but missing
In some non-standard implementations, the server may respond AUTH_USER_PASS
even if's not listed in available authentication methods. (it should
respond AUTH_NO_ACCEPTABLE per standards)
2017-04-22 21:48:41 +08:00
facfd79f9a [azubu] Remove extractor as the site is gone (closes #12813) 2017-04-22 21:20:25 +08:00
3110bb937d [porn91] Fix extraction (closes #12814) 2017-04-22 21:16:36 +08:00
cb2520802d [extractor/common] Improve m3u8 extraction (closes #12211)
* Extract m3u8 parsing to separate method
* Improve rendition groups extraction
* Build stream name according stream GROUP-ID
* Ignore reference to AUDIO group without URI when stream has no CODECS
+ Add test coverage for parsing m3u8 from #11507, #11995, #12211 and twitch vod
2017-04-22 07:01:00 +07:00
f779958250 [vidzi] Fix extraction (closes #12793) 2017-04-21 23:37:06 +07:00
8abc7dca39 [amp] extract error message(closes #12795) 2017-04-20 05:16:41 +01:00
ea0c2f219c [downloader/fragment] use a general file to store fragment download context 2017-04-19 18:53:15 +01:00
481ef51e23 [brightcove] PEP 8 2017-04-19 21:47:03 +07:00
5b995f713b [utils] add support for ttml styles 2017-04-19 14:38:40 +01:00
75a2485407 [fragment,hls,f4m,dash,ism] improve fragment downloading
- resume immediately
- no need to concatenate segments and decrypt them on every resume
- no need to save temp files for segments

and for hls downloader:
- no need to download keys for segments that already downloaded
2017-04-19 11:46:07 +01:00
58f6ab72ed [odnoklassniki] update tests 2017-04-19 00:16:55 +01:00
2dc48df5bc [xfileshare] Add support for gorillavid.com and daclips.com (closes #12776) 2017-04-18 23:58:37 +07:00
18848d226a [instagram] Fix extraction (closes #12777) 2017-04-18 22:40:26 +07:00
a32a9a7ef5 [extractor/common] Add support multiple getters in try_get 2017-04-18 22:39:58 +07:00
bae1404893 [extractor/common] Add support for video of WebPage context in _json_ld (closes #12778) 2017-04-18 22:21:38 +07:00
06d0ad9a4e [brightcove] Support URLs with bcpid instead of playerID
Fixes #12482
2017-04-18 23:04:22 +08:00
f631b55791 [brightcove] Fix _extract_url (closes #12782) 2017-04-18 21:46:25 +07:00
bf1b87cd91 [common] Relax JWPlayer regex and remove duplicate urls(#12768) 2017-04-17 08:48:24 +01:00
1c35b3da44 [odnoklassniki] extract m3u8 formats 2017-04-16 21:27:08 +01:00
16a09aefe3 release 2017.04.17 2017-04-17 00:46:32 +07:00
fa7a6e6de6 [ChangeLog] Actualize 2017-04-17 00:43:56 +07:00
ab87c26009 [extractor/generic] Add test for #12761 2017-04-17 00:36:59 +07:00
91bc57e4c5 [limelight] Add support for channels and channels lists embedded using LimelightEmbeddedPlayerFlash (#12761) 2017-04-17 00:33:47 +07:00
e5d39886ec [limelight] Improve embeds extraction (closes #12761)
* Move extraction code to extractor
* Add extraction for LimelightEmbeddedPlayerFlash embeds
* Extract multiple video
2017-04-17 00:23:16 +07:00
751c89a27d [itv] Extract series metadata 2017-04-16 23:19:20 +07:00
c2d7d76efd [itv] Fix rtmp formats (#12759) 2017-04-16 23:15:24 +07:00
f67177cae8 [itv] Use native hls 2017-04-16 21:52:45 +07:00
5935ef3c5d [itv] Lower preference for rtmp formats (closes #12759) 2017-04-16 21:52:07 +07:00
47e0cef46e [openload] rewrite extractor 2017-04-16 00:34:34 +02:00
1183e22c7e Credit @triple-j for extracting more metadata from go90.com(#12721) 2017-04-15 23:19:23 +01:00
72950c4dce Merge pull request #12752 from triple-j/go90_improvements_pull_request
[go90] Improve extraction
2017-04-15 23:48:13 +02:00
334f41e0d8 [go90] Improve extraction
- add metadata for 'series', 'episode', 'season', 'season_id',
  'season_number', and 'episode_number'
- integrate series title into the title
- extract subtitles  (fallback to `vtt` if the subtitle file type
  detection fails as that is the most likely extension)
2017-04-15 16:32:00 -05:00
022250a594 release 2017.04.16 2017-04-16 01:49:02 +07:00
76b5f99617 [ChangeLog] Actualize 2017-04-16 01:47:58 +07:00
d35dc344af [YoutubeDL] Apply expand_path after output template substitution
os.path.expandvars translates '%%' into '%' making output template invalid.
Before: '%%(ext)s' -(expand path)-> '%(ext)s' -(outtmpl subst.)-> 'mp4'
After: '%%(ext)s' -(outtmpl subst.)-> '%(ext)s' -(expand path)-> '%(ext)s'
2017-04-16 01:39:07 +07:00
51350db5a3 [test_YoutubeDL] Add test for #11163 2017-04-16 01:14:05 +07:00
0563f7ac6e [YoutubeDL] Propagate overridden metadata to IE results of type url (closes #11163) 2017-04-16 00:57:36 +07:00
413c1f8e2f [extractor/generic] Extract RSS entries as url_transparent (#11163) 2017-04-16 00:57:32 +07:00
8068296276 [streamango] Improve extraction (closes #12643) 2017-04-15 21:50:51 +07:00
4db79fa1bc [streamango] Add extractor 2017-04-15 21:50:46 +07:00
b2a19e3829 [wsj] Improve and modernize (closes #12558) 2017-04-15 20:59:05 +07:00
3266d08af2 [wsj:article] Add extractor 2017-04-15 20:58:57 +07:00
0254f93b08 [brightcove] Validate ambiguous embeds' URLs (closes #9163, closes #12005, closes #12178, closes #12480) 2017-04-15 20:31:49 +07:00
40158f55c9 [brightcove] Relax video tag embeds extraction
BrightcoveNewIE.extract_urls(): Handle player data when it is all in
attributes of the <video> tag, and also the id is
data-brightcove-video-id not data-video-id.

Add test to generic extractor, note HEAD gives 404, no checksum computed(?).
2017-04-15 20:31:43 +07:00
3dfceb286c [udemy] Add support for react rendition (closes #12744) 2017-04-15 03:26:19 +07:00
57e2cb6c3a [curiositystream] PEP 8 2017-04-15 02:33:26 +07:00
6f1b2374af [canalplus] PEP 8 2017-04-15 02:33:12 +07:00
d24bd4ffdb release 2017.04.15 2017-04-15 01:39:33 +07:00
b9859e2735 [ChangeLog] Actualize 2017-04-15 01:37:26 +07:00
34c3bbe1bd [youku] Keep old fileid extraction code as fallback (#12741) 2017-04-15 01:28:02 +07:00
e5106ba402 [youku] Fix fileid extraction 2017-04-14 13:23:02 -05:00
38d2f8325f release 2017.04.14 2017-04-14 00:31:49 +07:00
6f4dd6667b [ChangeLog] Actualize 2017-04-14 00:29:36 +07:00
95728fda70 [aenetworks] Add support for specials (closes #12723) 2017-04-13 23:39:55 +07:00
3f7409f745 [hbo] extract m3u8 formats 2017-04-13 12:28:22 +01:00
b2fff30817 [go90] Add new extractor(closes #10127) 2017-04-13 12:28:22 +01:00
f59746372a [downloader/hls] add basic support for EXT-X-BYTERANGE tag(#10955) 2017-04-13 12:28:22 +01:00
34d98cc411 [tv2hu] Fix invalid escape sequence on Py3.6 2017-04-13 16:52:52 +08:00
40fcba5edb improve coding style 2017-04-12 20:38:43 +01:00
e4d74e2778 [tv2hu] improve extraction 2017-04-12 19:44:32 +01:00
3ef1d0c733 [tv2hu] Add new extractor 2017-04-12 19:36:02 +01:00
3962260b7d Credit @SkiTheSlicer for Verizon adobe pass support (#12721) 2017-04-12 16:54:53 +01:00
0ee79a378a [generic] exclude urls with xml ext from valid video urls(closes #10768)(closes #11654) 2017-04-12 16:22:08 +01:00
79a51069e5 [adobepass] improve comcast and verison login code(closes #10803) 2017-04-12 15:19:58 +01:00
a9a346535b [adobepass] Add Verizon support (#10803) 2017-04-12 01:18:57 -04:00
89beedd31f [youtube] improve m3u8 format extraction 2017-04-11 21:55:59 +01:00
e109f1ff43 [afreecatv] Fix extraction for videos with different key layout (closes #12718) 2017-04-12 02:17:34 +07:00
d23028a8fb [youtube] Remove explicit preference for audio-only and video-only formats
In order not to break sorting when new formats appear
2017-04-11 22:41:48 +07:00
6214611a4a [canalplus] Bypass geo restriction 2017-04-11 22:30:15 +07:00
1730878167 release 2017.04.11 2017-04-11 02:17:53 +07:00
689cd458a6 [ChangeLog] Actualize 2017-04-11 02:16:20 +07:00
6b9466de2f [afreecatv] Fix extraction (closes #12706) 2017-04-11 02:05:53 +07:00
61568e50cf [generic] Add support for <object> youtube embeds (closes #12637) 2017-04-11 00:04:32 +07:00
364a69e8c6 [test_download] Fix testing playlists with single video and add comments 2017-04-11 00:01:02 +07:00
6240925b40 [bbccouk] Treat bitrate as audio+video bitrate in media selector 2017-04-10 22:56:22 +07:00
964744af95 [bbccouk] Skip unrecognized formats in media selector (#12701) 2017-04-10 22:53:51 +07:00
1af959ef9f [bbccouk] Add support for https protocol in media selector (closes #12701) 2017-04-10 22:53:06 +07:00
a206ef62df [curiositystream] fix extraction(closes #12638) 2017-04-10 13:50:08 +01:00
3f2ce6896a [adn] update subtitle decryption key 2017-04-09 12:33:29 +01:00
a6f7263cf4 [chaturbate] Fix extraction (closes #12665) 2017-04-09 01:39:40 +07:00
4372436504 release 2017.04.09 2017-04-09 00:01:28 +07:00
eb8cc8ea3b [ChangeLog] Actualize 2017-04-08 23:59:17 +07:00
41b263ac8a [canvas] Add IE_DESC (closes #12478) 2017-04-08 22:45:45 +07:00
ca8fca9d9d [vrt] Add IE_DESC (closes #12477) 2017-04-08 22:44:31 +07:00
e129fa0846 [vier] Add IE_DESC 2017-04-08 22:43:29 +07:00
2bd875edfe [medici] Add extractor (closes #3406) 2017-04-08 22:38:37 +07:00
95152630db [rbmaradio] Add support for redbullradio.com URLs 2017-04-08 21:39:07 +07:00
04e431cf97 [npo:live] Improve (closes #12555) 2017-04-08 21:31:22 +07:00
1591ba258a [npo:live] Add support for default url 2017-04-08 21:30:38 +07:00
29c6726646 [mixcloud] Fix view count extraction and modernize 2017-04-08 21:11:08 +07:00
a66e25859a [mixcloud:playlist] Relax title extraction and fix description extraction (closes #12582) 2017-04-08 21:04:09 +07:00
c93c0fc2fd [mixcloud:playlist] Fix title extraction 2017-04-08 20:47:07 +07:00
90e3f18fc1 [thesun] Extract playlists (closes #11298, closes #12674) 2017-04-08 20:08:31 +07:00
5f3e0b69ef [TheSun] Add new extractor 2017-04-08 19:54:04 +07:00
28b674ca23 [ceskateleveize:porady] Add extractor (closes #7411, closes #12645) 2017-04-08 19:46:42 +07:00
e18f1da97a [ceskateleveize] Improve extraction and remove URL replacement hacks 2017-04-08 19:41:14 +07:00
78280352ca [kaltura] Cleanup regexes from redundant escaping 2017-04-08 16:48:27 +07:00
a01825a541 [kaltura] Add support for iframe embeds 2017-04-08 16:40:57 +07:00
f8f2da25ab [wshh] Extract html5 entries and delegate to generic extractor (closes #12676) 2017-04-08 16:01:56 +07:00
4c03973296 [airmozilla] Fix extraction (closes #12670) 2017-04-08 15:39:58 +08:00
60e5016199 [test_download] Remove unused import 2017-04-08 14:19:01 +07:00
c4d6fc6d65 [test_subtitles] Fix raiplay test 2017-04-08 14:11:20 +07:00
1b3feca0a7 [raiplay] Extract subtitles 2017-04-08 14:11:03 +07:00
80b2fdf9ac [test_download] Match info dicts against tests before matching info file 2017-04-08 14:10:12 +07:00
3bef10a50c [test_download] typo in comment 2017-04-08 13:07:25 +07:00
a84da06f49 [test_download] Improve diagnostic on wrong 'id' 2017-04-08 13:07:25 +07:00
3461f5db06 [xfileshare] Add support for vidlo.us (closes #12660) 2017-04-06 23:39:09 +07:00
0378b8b917 [xfileshare] Add support for vidbom.com (closes #12661) 2017-04-06 23:38:16 +07:00
7f04386b89 [aenetworks] Add more video URL regexes (closes #12657) 2017-04-06 02:36:48 +07:00
fac39cccd4 [odnoklassniki] fix format sorting for 1080p quality 2017-04-03 23:39:56 +01:00
b68e00b08a [rtl2] add support for you.rtl2.de(closes #10257) 2017-04-03 21:36:35 +01:00
2ab0bfcd81 [vshare] Add extractor (closes #12278) 2017-04-04 03:05:18 +07:00
b022f4f600 release 2017.04.03 2017-04-03 03:53:55 +07:00
e2435ba5f3 [ChangeLog] Actualize 2017-04-03 03:52:44 +07:00
a9bb61a425 [discoveryvr] Add new extractor(closes #12578) 2017-04-02 09:22:09 +01:00
dbf70c489f [tv5mondeplus] clean description and use stable id 2017-04-02 00:26:48 +01:00
61e2331ad8 [tv5mondeplus] Add new extractor(closes #11386) 2017-04-01 23:49:40 +01:00
fd47550885 [extractor/common] Add coding cookie 2017-04-02 04:42:10 +07:00
4457823dda [extractor/common] Move censorship checks to a separate method and add check for just another ISP 2017-04-02 03:57:44 +07:00
b3633fa0ce [pericope] Add support for pscp.tv URLs 2017-04-02 03:20:28 +07:00
b56e41a701 release 2017.04.02 2017-04-02 02:39:15 +07:00
a76c25146a [ChangeLog] Actualize 2017-04-02 02:37:18 +07:00
361f293ab8 [rai] Skip not found content item id 2017-04-02 02:24:13 +07:00
b8d8cced9b [rai] Improve extraction (closes #11790)
* Fix georestriction detection
* Detect live streams
+ Extract relinker metadata
* Improve ContentItem detection
+ Extract series metadata
* Fix tests
2017-04-02 02:14:42 +07:00
51342717cd [rai] Fix extraction 2017-04-02 02:10:53 +07:00
48ab554feb [vrv] add support for series pages 2017-04-01 18:09:36 +01:00
a6f3a162f3 [limelight] improve extraction for audio only formats 2017-04-01 15:35:39 +01:00
91399b2fcc [funimation] fix extraction(closes #10696)(#11773) 2017-04-01 13:33:04 +01:00
eecea00d36 [xfileshare] Add support for vidabc.com (closes #12589) 2017-04-01 18:56:35 +07:00
2cd668ee59 [xfileshare] Improve extraction and extract hls formats 2017-04-01 18:55:48 +07:00
ca77b92f94 [crunchyroll] pass geo verifcation proxy 2017-04-01 09:33:23 +01:00
e97fc8d6b8 [cwtv] extract ISM formats 2017-04-01 07:50:24 +01:00
be61efdf17 [tvplay] Bypass geo restriction 2017-04-01 07:26:40 +01:00
77c8ebe631 [vrv] Add new extractor 2017-03-31 23:29:23 +01:00
7453999580 [packtpub] Add extractor (closes #12610) 2017-04-01 00:25:27 +07:00
1640eb0961 [YoutubeDL] Return early when extraction of url_transparent fails 2017-03-31 23:57:35 +07:00
3e943cfe09 [generic] pass base_url to _parse_jwplayer_data 2017-03-31 14:54:06 +01:00
82be732b17 [adn] Add new extractor 2017-03-31 12:24:23 +01:00
639e5b2a84 [allocine] Extract more metadata 2017-03-29 04:43:12 +07:00
128244657b [allocine] Fix extraction 2017-03-29 05:23:20 +08:00
12ee65ea0d [options] Mention ISM for --fragment-retries and --skip-unavailable-fragments 2017-03-28 23:35:48 +07:00
aea1dccbd0 [openload] fix extractor 2017-03-29 00:00:09 +08:00
9e691da067 release 2017.03.26 2017-03-26 08:11:40 +07:00
82eefd0be0 [ChangeLog] Actualize 2017-03-26 23:39:12 +07:00
f7923a4c39 [ChangeLog] Update after #12307 2017-03-26 22:07:12 +08:00
cc63259d18 Merge pull request #12307 from rndusr/fix/str-item-assignment
Fix "'str' object does not support item assignment"
2017-03-26 21:51:09 +08:00
2bfaf89b6c [downloader/hls] move check for m3u8 live streams to get_suitable_downloader 2017-03-25 23:07:05 +01:00
4f06c1c9fc Merge branch 'master' of github.com-rndusr:rg3/youtube-dl into fix/str-item-assignment 2017-03-25 21:36:59 +01:00
942b44a052 [test_compat] Do not use dash in env variables' names 2017-03-26 03:24:25 +07:00
a426ef6d78 [test_utils] Do not use dash in env variables' names 2017-03-26 03:22:48 +07:00
41c5e60dd5 [test_utils] Fix expand_path tests 2017-03-26 03:07:56 +07:00
d212c93d16 [pluralsight] PEP 8 2017-03-26 02:34:25 +07:00
15495cf3e5 [franceculture] PEP 8 2017-03-26 02:32:46 +07:00
5b7cc56b05 [atresplayer] PEP 8 2017-03-26 02:32:14 +07:00
590bc6f6a1 Use expand_path where appropriate (closes #12556) 2017-03-26 02:31:16 +07:00
51098426b8 [utils] Introduce expand_path 2017-03-26 02:30:10 +07:00
c73e330e7a _find_jwplayer_data() returns dict or None
This simplifies code for callers of `_find_jwplayer_data()` which no longer have
to run `_parse_json()` on the return value.

It also makes sure that `_find_jwplayer_data()` returns either a `dict` or
`None` and nothing else.
2017-03-25 19:38:30 +01:00
fb4fc44928 [downloader/hls] immediately delegate downloading to ffmpeg in case live stream 2017-03-25 19:38:23 +01:00
03486dbb01 Add test for JWPlayer where config is passed as variable 2017-03-25 19:37:45 +01:00
51ef4919df [afreecatv] Fix extraction (closes #12179) 2017-03-26 01:32:07 +08:00
d66d43c554 [atvat] Add new extractor(closes #5325) 2017-03-25 18:13:58 +01:00
610a6d1053 [atresplayer] Do not extract ISM formats
As per @remitamine: the ISM downloader does not support videos served from wowza servers(it will produce broken files)
2017-03-25 21:40:54 +07:00
c6c22e984d [test_download] Print additional IEs in summary output 2017-03-25 22:36:40 +08:00
d97729c83a [fox] remove unused import 2017-03-25 14:28:53 +01:00
7aa0ee321b [fox] Add metadata extraction
Add series, season number, episode number and episode.
2017-03-25 21:12:25 +08:00
e8e4cc5a6a [generic] Replace LazyYT test with skiplagged
discourse.ubuntu.com has gone away, repalce with skiplagged.com.
Be nice to have a non-frontpage URL that might be more stable,
though I don't have one. Maybe this should move to html
in test/test_InfoExtractor.py?
2017-03-25 19:53:32 +07:00
c7301e677b [atresplayer] Extract DASH and ISM formats 2017-03-25 18:03:46 +07:00
048086920b [atresplayer] Extract HD manifest 2017-03-25 17:52:04 +07:00
1088d76da6 [atresplayer] Fix login error detection 2017-03-25 17:47:35 +07:00
31a1214076 [franceculture] fix extraction(closes #12547) 2017-03-25 07:04:48 +01:00
d0ba55871e [youtube] Improve _VALID_URLs (closes #12538) 2017-03-25 01:18:33 +07:00
54b960f340 [generic] Do not follow redirects to the same URL 2017-03-24 00:45:24 +07:00
a3ccd6bd11 release 2017.03.24 2017-03-24 00:24:23 +07:00
7963b6cba8 [ChangeLog] Actualize 2017-03-24 00:19:58 +07:00
bea7af6947 [channel9] Remove expired comment and sort imports 2017-03-23 23:58:12 +07:00
a5d783f525 [channel9] Extract more formats 2017-03-23 23:47:43 +07:00
d0572557c2 [ninecninemedia] remove mp4 url extraction request 2017-03-23 13:53:07 +01:00
52d5ecabd5 [bellmedia] add support for etalk.ca(closes #12447) 2017-03-23 13:52:45 +01:00
b0f7f21cb9 [channel9] fix extraction(closes #11323) 2017-03-23 09:22:37 +01:00
579c99a284 [cloudy] Fix extraction (closes #12525) 2017-03-22 23:48:06 +07:00
ca5ed022e9 [hbo] add support for free episode urls and new formats extraction(closes #12519) 2017-03-22 17:28:53 +01:00
391d076d7c [condenast] Fix extraction and style (closes #12526) 2017-03-22 23:22:14 +07:00
c183e14f89 [viu] Relax _VALID_URL (closes #12529) 2017-03-22 22:26:59 +07:00
093dad9e25 release 2017.03.22 2017-03-22 02:36:50 +07:00
e8686e51d7 [ChangeLog] Actualize 2017-03-22 02:35:09 +07:00
8e5a7c5e67 [pluralsight] Omit module title from video title (closes #12506) 2017-03-22 02:28:04 +07:00
e1e35d1ac6 [pornhub] Improve extraction and style (closes #12515) 2017-03-22 01:59:27 +07:00
21fbf0f955 [pornhub] Decode obfuscated video URL (closes #12470) 2017-03-22 01:51:45 +07:00
97952bdb78 [generic] Add test for Senate ISVP iframe embed 2017-03-22 01:12:14 +08:00
8a8cc339b6 [senateisvp] Allow https URL scheme for embeds 2017-03-20 23:35:13 +08:00
957f453429 [Openload.co] Fixed Extraction
They did it again. just a minor change though. here's quick fix
2017-03-20 16:15:00 +08:00
0e9a73e612 release 2017.03.20 2017-03-20 00:07:57 +07:00
0ecdd3adbd [ChangeLog] Actualize 2017-03-20 00:03:58 +07:00
9487ce03e9 [YoutubeDL] Allow multiple input URLs to be used with stdout as output template 2017-03-19 23:59:40 +07:00
45e6ad21b4 Credit @mrBliss for vtm (#11912) 2017-03-19 23:48:02 +07:00
68220649fa [ChangeLog] Update after #12099 2017-03-19 20:42:17 +08:00
46b18f2349 [BostonGlobe] New. Nonstandard version of Brightcove.
Has a "data-brightcove-video-id" instead of a "data-video-id," otherwise
pretty much just Brightcove. Except the Globe isn't all Brightcove
videos, so fallback to Generic, too.

Also, abstract playlist_from_matches() from generic.py to common.py, and use
it here.

History of these changes can be found in
51170427d4b1143572a498dedaee61863a5b2c5b.
2017-03-19 20:40:31 +08:00
772b5ff57f [toongoggles] Add new extractor(closes #12171) 2017-03-19 00:45:38 +01:00
f68ef1e2ab [medialaan] Remove unrelated test 2017-03-18 23:23:47 +07:00
febfe1e262 [adobepass] Detect and output error on authz token extraction (#12472) 2017-03-18 06:21:31 +07:00
5f0daab1ca [openload] Fix extraction 2017-03-18 07:02:55 +08:00
2a721cdff2 [medialaan] Fix and improve extraction (closes #11912) 2017-03-18 05:58:54 +07:00
e7a51a4c02 [vtm] Add extractor (closes #9974)
Implementation of the approach described in #9974.
2017-03-18 00:27:04 +07:00
3e5856d860 [discoverynetworks] add support for more domains and bypass geo restiction 2017-03-17 09:53:44 +01:00
ea883a687c [openload] Fix extraction (closes #10408)
Thanks to @makgun02

Ref: http://pastebin.com/raw/JX9gHFUz
2017-03-17 15:22:34 +08:00
7f3590c43b [test_InfoExtractor] Add some realworld tests for _extract_jwplayer_data 2017-03-17 00:00:01 +07:00
7d539ee10a release 2017.03.16 2017-03-16 22:42:12 +07:00
6ad476079d [ChangeLog] Actualize 2017-03-16 22:39:48 +07:00
0efbc6b56d [options] Mention flac support and sort alphabetically among the audio formats 2017-03-16 12:54:47 +01:00
21bfcd3d6e [postprocessor/ffmpeg] Add support for flac
Requested at http://stackoverflow.com/q/42828041/35070
2017-03-16 12:50:45 +01:00
b51dc9db0e [extractor/common] Extract SMIL formats from jwplayer 2017-03-16 03:30:53 +07:00
a309684285 [extractor/generic] Add forgotten return for jwplayer formats 2017-03-16 03:28:01 +07:00
ba448445b8 [redbull] improve extraction
- extract 1080p quality
- correct ttml subtitle ext
- catch api errors
- reduce request size
2017-03-15 01:40:54 +01:00
5db83d79bf release 2017.03.15 2017-03-15 02:01:24 +07:00
2a751e137f [ChangeLog] Actualize 2017-03-15 02:00:10 +07:00
398887b4c0 [Openload] Fixed Extraction
They did changed it again.
2017-03-14 14:03:52 +08:00
66bf351f80 [facebook] Make title optional (closes #12443) 2017-03-14 00:38:07 +07:00
9d08963022 [telecinco] Add test for #12430 2017-03-13 22:41:28 +07:00
e313d209c2 [mitele] Add support for ooyala videos (closes #12430) 2017-03-13 22:39:15 +07:00
ff9d509d20 [openload] Fix extraction
Just a minor fix for openload
2017-03-13 04:22:35 +08:00
c1795ca6c8 [streamable] Update API URL 2017-03-13 02:51:59 +08:00
8c99623259 [crunchyroll] Extract season name 2017-03-12 12:18:10 +08:00
57b0ddb35f [discoverygo] Actualize test 2017-03-11 23:21:08 +07:00
a28f8d7396 [discoverygo] Bypass geo restriction 2017-03-11 23:18:42 +07:00
7049799470 [discoverygo:playlist] Add extractor (closes #12424) 2017-03-11 23:16:51 +07:00
4605c94d1a [__init__] Fix missing subtitles if --add-metadata is used (#12423)
The previous fix for #5594 is incorrect
2017-03-11 19:37:45 +08:00
a8e687a4da release 2017.03.10 2017-03-10 23:26:28 +07:00
f9e5c92c94 [ChangeLog] Actualize 2017-03-10 23:23:24 +07:00
c2ee861c6d [extractor/generic] Make title optional for jwplayer embeds (closes #12410) 2017-03-10 23:16:53 +07:00
bd34c32bd7 [wdr] Actualize comment 2017-03-10 23:07:36 +07:00
f802c48660 [wdr:maus] Fix extraction and update tests 2017-03-10 23:59:32 +08:00
76bee08fe7 [prosiebensat1] Improve title extraction and add test 2017-03-09 23:42:07 +07:00
2913821723 [prosiebensat1] Improve title extraction (closes #12318) 2017-03-10 00:18:37 +08:00
0e7f9a9b48 [dplayit] Relax playback info URL extraction 2017-03-08 21:30:30 +07:00
0cf2352e85 [dplayit] Separate and rewrite extractor and bypass geo restriction (closes #12393) 2017-03-08 21:20:01 +07:00
0f6b87d067 [miomio] Fix extraction
Closes #12291
Closes #12388
Closes #12402
2017-03-08 19:46:58 +08:00
d7344d33b1 [telequebec] Fix description extraction and update test (closes #12399) 2017-03-08 18:25:59 +07:00
b08cc749d6 [openload] Fix extraction 2017-03-08 06:01:27 +08:00
b68a812ea8 [extractor/generic] Add test for brigthcove UUID-like videoPlayer 2017-03-07 23:00:21 +07:00
2e76bdc850 [brightcove:legacy] Relax videoPlayer validation check (closes #12381) 2017-03-07 22:59:33 +07:00
fe646a2f10 [twitch] PEP8 2017-03-07 15:34:06 +08:00
9df53ea36e Credit @puxlit for twitch 2fa (#11974) 2017-03-07 04:05:47 +07:00
d7d7f84c95 Credit @benages for redbull.tv (#11948) 2017-03-07 04:05:47 +07:00
dccd0ab35d release 2017.03.07 2017-03-07 03:59:22 +07:00
80146dcc6c [ChangeLog] Actualize 2017-03-07 03:57:54 +07:00
e30ccf7047 [soundcloud] Update client id (closes #12376) 2017-03-06 23:05:38 +07:00
54a3a8827b [__init__] Metadata should be added after conversion
Fixes #5594
2017-03-06 18:09:12 +08:00
92cb5763f4 [ChangeLog] Update after #12357 2017-03-06 18:04:19 +08:00
da92da4b88 Openload fix extraction (#12357)
* Fix extraction
2017-03-06 18:00:17 +08:00
1664702626 release 2017.03.06 2017-03-06 04:04:39 +07:00
3f116b189b [ChangeLog] Actualize 2017-03-06 04:01:21 +07:00
4b5de77bdb [utils] Process bytestrings in urljoin (closes #12369) 2017-03-06 03:57:46 +07:00
96182695e4 [drtv] Add geo countries to GeoRestrictedError 2017-03-06 03:23:42 +07:00
fc11ad3833 [drtv:live] Bypass geo restriction 2017-03-06 03:23:42 +07:00
d2b64e04b4 [addanime] Skip an invalid test 2017-03-06 00:35:04 +08:00
5dd376345b [tunepk] Add extractor (closes #12197, closes #12243) 2017-03-05 23:31:38 +07:00
1a2192cb90 [extractor/common] Pass arguments to _parse_jwplayer_formats and PEP8 2017-03-05 23:29:17 +07:00
0236cd0dfd [extractor/common] Improve height extraction and extract bitrate 2017-03-05 23:25:03 +07:00
ed0cf9b383 [extractor/common] Move jwplayer formats extraction in separate method 2017-03-05 23:22:27 +07:00
a50862b735 [downloader/external] Add missing import and PEP8 2017-03-05 10:24:29 +07:00
6d0fe752bf [external:ffmpeg] In test harness, limit to 10k download size
Otherwise, if you screw up a playlist test by including a playlist
dictionary key, you'll be there for eons while it downloads all the
files before erroring out.
2017-03-05 11:19:44 +08:00
afa4597618 release 2017.03.05 2017-03-05 02:23:08 +07:00
75027364ba [ChangeLog] Actualize 2017-03-05 02:22:02 +07:00
5316566edc [twitch] Use better naming and simplify (closes #11974) 2017-03-05 02:06:33 +07:00
c64c03be35 [twitch] Add basic support for two-factor authentication 2017-03-05 01:06:27 +07:00
bcefc59279 Credit @vierbergenlars for vijf.be (#12304) 2017-03-05 00:03:59 +07:00
6f211dc936 Credit @obilodeau for vrak (#11452) 2017-03-05 00:03:59 +07:00
f24c1e5584 Credit @TobiX for #9725 2017-03-05 00:03:59 +07:00
466274fe9a Credit @p2004a for vodpl (#12122) 2017-03-05 00:03:59 +07:00
30f8f142d4 Credit @ThomasChr for #12015 and #12245 2017-03-05 00:03:59 +07:00
a3ba8a7acf [vier] Add support for vijf.be
vier.be and vijf.be run on the same CMS and are property of the same company,
so the same extractor can be used for both of them.
2017-03-05 00:47:19 +08:00
054a587de8 [redbulltv] Improve extraction (closes #11948, closes #3919) 2017-03-04 23:28:21 +07:00
64b7ccef3e [redbulltv] Add extractor 2017-03-04 23:26:15 +07:00
6f4e4132d8 [douyutv] Switch to the PC API to escape the 5-min limitation
Thanks @spacemeowx2 for the algo.

Ref: https://gist.github.com/spacemeowx2/629b1d131bd7e240a7d28742048e80fc

Closes #12316
2017-03-04 23:23:18 +08:00
eb3079b6ce [generic] Add support for rutube embeds 2017-03-04 00:46:33 +07:00
bc82f22879 [rutube] Relax _VALID_URL 2017-03-04 00:42:51 +07:00
4d058c9862 [vrak] Improve and update test (closes #11452) 2017-03-03 23:58:16 +07:00
d16f27ca27 [brightcove:new] Add ability to smuggle geo_countries into URL 2017-03-03 23:58:03 +07:00
cbb127568a [vrak] Add extractor 2017-03-03 23:54:21 +07:00
d02d4fa0a9 [brightcove:new] Raise GeoRestrictedError 2017-03-03 22:49:48 +07:00
692fa200ca [go] Relax _VALID_URL (closes #12341) 2017-03-03 22:28:34 +07:00
9bae185ba6 [24video] Use original host for requests (closes #12339) 2017-03-03 22:16:00 +07:00
4d345bf17b [ruutu] Disable DASH formats (closes #12322)
Due to causing out of sync issue
2017-03-02 23:53:46 +07:00
250eea6821 release 2017.03.02 2017-03-02 22:33:22 +07:00
28d15b73f8 [ChangeLog] Actualize 2017-03-02 22:29:56 +07:00
11bb6ad1a5 [facebook] Fix extraction (closes #12323)
Almost all videos now use the pagelet type 'permalink_video_pagelet'
2017-03-02 20:51:24 +08:00
c9612c0487 [youtube] Mark errors about rental videos as expected
Closes #12324
2017-03-02 16:59:53 +08:00
af5049f128 [adobepass] Add Charter Spectrum (#11465)
Thanks @tv21 for the fix!
2017-03-02 02:15:51 +08:00
158af5242e [utils] Carry long doc string 2017-03-01 23:04:02 +07:00
40df485f55 [YoutubeDL] Don't sanitize identifiers (closes #12317) 2017-03-01 23:03:36 +07:00
4b8a984c67 [npo] Add support for audio 2017-03-01 22:21:13 +07:00
83e8fce628 [npo] Improve extraction and update tests 2017-03-01 22:14:46 +07:00
aa9cc2ecbf [npo] Adapt to app.php API (closes #12311) 2017-03-01 05:03:35 +07:00
1dc24093f8 release 2017.02.28 2017-02-28 23:59:22 +07:00
11bae9cdde [ChangeLog] Actualize 2017-02-28 23:49:24 +07:00
43b38424a9 [azmedien:showplaylist] Improve (closes #12160) 2017-02-28 23:37:54 +07:00
948519b35d [azmedien:showplaylist] Add support for all episodes playlists 2017-02-28 23:36:05 +07:00
87dadd456a [youtube:playlist] Recognize another playlist pattern (closes #11928, closes #12286) 2017-02-28 23:06:47 +07:00
7c4aa6fd6f [daisuki] Add subtitles (#4738) 2017-02-28 22:29:01 +08:00
9bd05b5a18 [daisuki] Add new extractor (closes #4738) 2017-02-28 22:19:26 +08:00
0a5445ddbe [utils] Add bytes_to_long() and long_to_bytes()
Used in daisuki.net (#4738)

Both are adapted from public domain PyCrypto:
https://github.com/dlitz/pycrypto/blob/master/lib/Crypto/Util/number.py
2017-02-28 22:10:31 +08:00
f48409c7ac [utils] Add pkcs1pad
Used in daisuki.net (#4738)
2017-02-28 22:10:31 +08:00
c9619f0a17 [aes] Add aes_cbc_encrypt
Used in daisuki.net (#4738)
2017-02-28 22:10:31 +08:00
f4c68ba372 [douyu] Fix extraction and update _TESTS
They've switched from flv to hls

Closes #12301
2017-02-28 21:41:03 +08:00
ef48a1175d release 2017.02.27 2017-02-27 23:26:07 +07:00
c6184bcf7b [ChangeLog] Actualize 2017-02-27 23:24:03 +07:00
18abb74376 [npo] Relax _VALID_URL for zapp.nl 2017-02-27 23:13:51 +07:00
dbc01fdb6f [hetklokhuis] Fix IE_NAME 2017-02-27 23:10:29 +07:00
f264c62334 [npo] Add support for zapp.nl 2017-02-27 23:10:00 +07:00
0dc5a86a32 [npo] Add support for hetklokhuis.nl (closes #12293) 2017-02-27 22:43:19 +07:00
0e879f432a [youtube:channel] Remove duplicate test 2017-02-27 22:22:43 +07:00
892b47ab6c [scivee] Remove extractor (#9315)
The Wikipedia page is changed from active to down:
https://en.wikipedia.org/w/index.php?title=SciVee&diff=prev&oldid=723161154

Some other interesting bits:

$ nslookup www.scivee.tv
Server:         8.8.8.8
Address:        8.8.8.8#53

Non-authoritative answer:
www.scivee.tv   canonical name = scivee.rcsb.org.
Name:   scivee.rcsb.org
Address: 132.249.231.211

$ nslookup rcsb.org
Server:         8.8.8.8
Address:        8.8.8.8#53

Non-authoritative answer:
Name:   rcsb.org
Address: 132.249.231.77

Both IPs are from UCSD. I guess it's maintained by a lab and they don't
maintain it anymore.
2017-02-27 21:34:33 +08:00
fdeea72611 [cda] Decode URL (fixes #12255) 2017-02-26 22:05:52 +08:00
xbe
7fd4655256 [crunchyroll] Extract uploader name that's not a link
Provide the Crunchyroll extractor with the ability to extract uploader
names that aren't links. Add a test for this new functionality.
This fixes #12267.
2017-02-26 19:08:10 +08:00
fd5c4aab59 [youtube] Raise GeoRestrictedError 2017-02-26 16:52:40 +07:00
8878789f11 [dailymotion] Raise GeoRestrictedError 2017-02-26 16:52:40 +07:00
a5cf17989b [MDR] Relax _VALID_URL and playerURL matching and update _TESTS
Ref: #12169
2017-02-26 17:24:54 +08:00
b3aec47665 [tvigle] Raise GeoRestrictedError 2017-02-25 23:27:45 +07:00
9d0c08a02c [vevo] Fix videos with the new streams/streamsV3 format (closes #11719) 2017-02-26 00:15:49 +08:00
e498758b9c [freshlive] Fix issues and improve (closes #12175) 2017-02-25 22:56:42 +07:00
5fc8d89361 [freshlive] Add extractor 2017-02-25 22:55:17 +07:00
d374d943f3 [downloader/common] Limit displaying 2 digits after decimal point in sleep interval message 2017-02-25 20:59:04 +07:00
103f8c8d36 [xhamster] Capture and output videoClosed error (#12263) 2017-02-25 20:38:21 +07:00
922ab7840b [etonline] Add extractor (closes #12236) 2017-02-25 20:16:40 +07:00
831217291a [compat] Use try except for compat_numeric_types 2017-02-25 19:44:50 +07:00
db182c63fb [njpwworld] Add new extractor (closes #11561) 2017-02-25 18:44:39 +08:00
eeb0a95684 [extractor/common] Add 'preference' to _parse_html5_media_entries
Some websites, like NJPWorld, put different qualities on different
player pages.
2017-02-25 18:40:05 +08:00
231bcd0b6b [amcnetworks] Relax _VALID_URL (#12127) 2017-02-25 02:51:53 +07:00
204efc8509 release 2017.02.24.1 2017-02-24 21:59:39 +07:00
5d3a51e1b9 [ChangeLog] Actualize 2017-02-24 21:57:39 +07:00
ad3033037c [noco] Modernize 2017-02-24 21:51:56 +07:00
f3bc281239 [noco] Swtich login URL to https (closes #12246) 2017-02-24 21:48:34 +07:00
441d7a32e5 [thescene] Extract more metadata 2017-02-24 21:22:29 +07:00
51ed496307 [thescene] Fix extraction (closes #12235) 2017-02-24 22:08:45 +08:00
68f17a9c2d [tubitv] use geo bypass mechanism 2017-02-24 12:27:56 +01:00
39e7277ed1 [openload] fix extraction(closes #10408) 2017-02-24 11:21:58 +01:00
42dcdbe11c [ivi] Raise GeoRestrictedError 2017-02-24 10:54:39 +07:00
6b097cff27 release 2017.02.24 2017-02-24 06:09:15 +07:00
f2f7961820 [ChangeLog] Actualize 2017-02-24 06:07:41 +07:00
be5df5ee31 Suppress help for all deprecated options and print warning when used 2017-02-24 06:04:27 +07:00
f2980fddeb [lynda:course] Add webpage extraction fallback (closes #12238) 2017-02-24 05:01:31 +07:00
0f57447de7 [postprocessor/ffmpeg] Add mising space (closes #12232) 2017-02-24 04:56:58 +07:00
19f3821821 [devscripts/make_lazy_extractors] Fix making lazy extractors on python 3 under Windows 2017-02-24 02:09:51 +07:00
8e1409fd80 [go] sign all uplynk urls and use geo bypass only for free videos(closes #12087)(closes #12210) 2017-02-23 18:42:06 +01:00
050f143c12 [README.md] Clarify sequence types in output template and document numeric string formatting operations 2017-02-23 23:00:13 +07:00
fafc2bf5a9 [options] Deprecate --autonumber-size 2017-02-23 22:11:16 +07:00
b3175982c3 [YoutubeDL] Add more numeric fields for NA substitution in outtmpl 2017-02-23 22:01:57 +07:00
89db639dfe [YoutubeDL] Rewrite outtmpl for playlist_index and autonumber for backward compatibility 2017-02-23 22:01:09 +07:00
d0d9ade486 [YoutubeDL] Add support for string formatting operations in output template 2017-02-23 22:57:53 +08:00
28572a1a0b [compat] Add compat_numeric_types 2017-02-23 22:57:53 +08:00
0f3d41b44d [devscripts/run_tests] Exclude youtube lists tests from core build 2017-02-23 19:48:54 +07:00
d5fd9a3be3 [skylinewebcams] Add extractor (closes #12221) 2017-02-23 18:45:38 +07:00
ada77fa544 [instagram] Add support for multi video posts (closes #12226) 2017-02-23 18:02:04 +07:00
9e03aa75c7 [crunchyroll] extract playlist entries ids 2017-02-23 11:57:18 +01:00
30eaa3a702 [mgtv] fix extraction 2017-02-23 11:57:05 +01:00
c59f703610 [sohu] raise GeoRestrictedError 2017-02-23 11:56:55 +01:00
bc61c80c14 [leeco] raise GeoRestrictedError and use geo bypass mechanism 2017-02-23 11:56:45 +01:00
345b24538b release 2017.02.22 2017-02-22 23:50:42 +07:00
63a29b6118 [ChangeLog] Actualize 2017-02-22 23:45:01 +07:00
b5869560a4 [crunchyroll] Fix descriptions with double quotes (closes #12124) 2017-02-23 00:08:45 +08:00
527ef85fe9 [dailymotion] Make comment count optional (closes #12209)
Not served anymore
2017-02-22 21:49:30 +07:00
58ad6995cd [vidzi] Add test for #12213 2017-02-22 21:29:53 +07:00
a86e416088 [vidzi] Add support for vidzi.cc 2017-02-22 22:28:09 +08:00
71e9577b94 [24video] Add support for 24video.tube (closes #12217) 2017-02-22 21:19:52 +07:00
0d427c8304 [setup] Actualize maintainer info 2017-02-22 01:51:27 +07:00
139d8ac106 [setup] Add python 3.6 classifier 2017-02-22 01:50:34 +07:00
abd29a2ced [crackle] use geo bypass mechanism 2017-02-21 19:37:26 +01:00
31615ac279 [viewster] use geo verifcation headers 2017-02-21 19:36:39 +01:00
fc320a40d9 Revert "[cbc] use geo bypass mechanism"
This reverts commit 86466a8b6f.
2017-02-21 18:14:55 +01:00
7345d6d465 [tfo] Improve geo restriction detection and use geo bypass mechanism 2017-02-21 17:52:50 +01:00
86466a8b6f [cbc] use geo bypass mechanism 2017-02-21 17:52:50 +01:00
33dc173cdc [telequebec] use geo bypass mechanism 2017-02-21 17:52:50 +01:00
3444844b04 [limelight] extract PlaylistService errors 2017-02-21 17:52:50 +01:00
8c6c88c7da release 2017.02.21 2017-02-21 23:48:24 +07:00
159aaaa9d0 [ChangeLog] Actualize 2017-02-21 23:46:58 +07:00
eea0716cae [extractor/common] Print origin country for fake IP 2017-02-21 23:14:33 +07:00
336a76551b [extractor/common] Do not quit _initialize_geo_bypass on empty countries 2017-02-21 23:09:41 +07:00
dc0a869e5e [extractor/common] Fix typo 2017-02-21 23:05:31 +07:00
e39b5d4ab8 [extractor/common] Allow calling _initialize_geo_bypass from extractors (#11970) 2017-02-21 23:00:43 +07:00
e469ab2528 [ninecninemedia] use geo bypass mechanism 2017-02-21 14:38:00 +01:00
890d44b005 [adobepass] add support for Time Warner Cable(closes #12191) 2017-02-20 19:00:40 +01:00
6926304472 [spankbang] Make uploader optional (closes #12193) 2017-02-21 00:54:43 +07:00
3ccdde8cb7 [extractor/common] Emphasize geo bypass APIs are experimental 2017-02-20 23:21:15 +07:00
da42ff0668 [iprima] Improve geo restriction detection and disable geo bypass 2017-02-20 23:17:19 +07:00
82f662182b [iprima] Modernize 2017-02-20 23:16:14 +07:00
2cc7fcd338 [commonmistakes] Disable UnicodeBOM extractor test for python 3.2 2017-02-20 03:06:52 +07:00
6d4c259765 [svt] PEP 8 2017-02-20 02:25:55 +07:00
c78dd35491 [nrk] PEP 8 2017-02-20 02:25:39 +07:00
8ffb8e63fe [prosiebensat1] Throw ExtractionError on unsupported page type (closes #12180) 2017-02-20 01:00:53 +07:00
983e9b7746 [nrk] Update _API_HOST and relax _VALID_URL 2017-02-20 00:59:31 +07:00
8936f68a0b [travis] Run tests in parallel
[test_download] Print test names in case of network errors

[test_download] Add comments for nose parameters

[test_download] Modify outtmpl to prevent info JSON filename conflicts

Thanks @jaimeMF for the idea.

[travis] Only download tests should be run in parallel
2017-02-19 21:26:35 +08:00
c58b7ffef4 [tv4] Bypass geo restriction and improve detection 2017-02-19 06:25:59 +07:00
f1a78ee4ef [tv4] Switch to hls3 protocol (closes #12177) 2017-02-19 06:16:00 +07:00
de64e23c56 [downloader/ism] Honor HTTP headers when downloading fragments 2017-02-19 04:18:36 +07:00
553f6dbac7 [downloader/dash] Honor HTTP headers when downloading fragments
For example, https://www.oppetarkiv.se/video/1196142/natten-ar-dagens-mor
2017-02-19 04:18:22 +07:00
0aa10994f4 [options] Move geo restriction related options to separate section 2017-02-19 05:10:08 +08:00
4248dad92b Improve geo bypass mechanism
* Rename options to preffixly match with --geo-verification-proxy
* Introduce _GEO_COUNTRIES for extractors
* Implement faking IP right away for sites with known geo restriction
2017-02-19 05:10:08 +08:00
0a840f584c Rename bypass geo restriction options 2017-02-19 05:10:08 +08:00
0016b84e16 Add faked X-Forwarded-For to formats' HTTP headers 2017-02-19 05:10:08 +08:00
18a0defab0 [utils] Make random_ipv4 return unicode string 2017-02-19 05:10:08 +08:00
5d3fbf77d9 [viki] Improve geo restriction detection 2017-02-19 05:10:08 +08:00
80b59020e0 [vgtv] Improve geo restriction detection 2017-02-19 05:10:08 +08:00
71631862f4 [srgssr] Improve geo restriction detection 2017-02-19 05:10:08 +08:00
89cc7fe770 [vbox7] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
04d906eae3 [svt] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
8ab8066cf0 [pbs] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
01b1aa9ff4 [ondemandkorea] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
ff4007891f [nrk] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
28200e654b [itv] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
e633f21a96 [go] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
d392005a79 [dramafever] Improve geo restriction detection and use geo bypass mechanism 2017-02-19 05:10:08 +08:00
773f291dcb Add experimental geo restriction bypass mechanism
Based on faking X-Forwarded-For HTTP header
2017-02-19 05:10:08 +08:00
bf5b9d859a [utils] Introduce YoutubeDLError base class for all youtube-dl exceptions 2017-02-19 05:10:08 +08:00
049a0f4d6d [brightcove:legacy] restrict videoPlayer value(closes #12040) 2017-02-18 21:08:40 +01:00
ac33accd96 [options] Mention quoted string literals for --match-filter 2017-02-18 23:59:26 +07:00
e84888b432 [tvn24] Improve extraction (closes #11679) 2017-02-18 23:34:09 +07:00
02d9b82a23 [tvn24] Add extractor 2017-02-18 23:33:49 +07:00
a2e3286676 [thisav] Add support for html5 media (closes #11771) 2017-02-18 20:21:53 +07:00
f75caf059e [metacafe] Improve (closes #10371) 2017-02-18 19:58:25 +07:00
bdabbc220c [metacafe] Bypass family filter
If you don't send this user=ffilter: false cookie, it will 301 redirect you to a page asking about it, and then the title check will fail.
2017-02-18 19:47:33 +07:00
70bcc444a9 [viceland] improve info extraction and update test 2017-02-18 09:52:43 +01:00
28e35f5070 release 2017.02.17 2017-02-17 23:59:56 +07:00
cf3704c132 [ChangeLog] Actualize 2017-02-17 23:48:30 +07:00
2c1f442c2b [options] Add missing spaces 2017-02-17 23:18:26 +07:00
bad4ccdb5d [heise] Improve (closes #9725) 2017-02-17 23:09:40 +07:00
db76c30c6e [heise] Support videos embedded in any article. 2017-02-17 22:55:53 +07:00
c2bde5d081 [ellentv] Improve 2017-02-17 22:45:51 +07:00
90fad0e74c [openload] Fix extraction (closes #12002) 2017-02-17 22:31:16 +07:00
d94badc755 [openload] Semifix extraction (closes #10408)
just updated the code. i don't do much python still i tried to convert my code. lemme know if there is any prob with it
2017-02-17 22:30:05 +07:00
fef51645d6 [theplatform] Recognize URLs with whitespaces (closes #12044) 2017-02-17 23:13:51 +08:00
4cead6a614 [einthusan] Relax _VALID_URL (closes #12141, closes #12159) 2017-02-17 22:02:01 +07:00
a4a554a793 [generic] Try parsing JWPlayer embedded videos (closes #12030) 2017-02-16 23:44:03 +08:00
b898f0a173 [elpais] Fix typo and improve extraction (closes #12139) 2017-02-16 04:57:42 +07:00
2480b056c1 release 2017.02.16 2017-02-16 00:10:04 +07:00
3aa25395aa [ChangeLog] Actualize 2017-02-16 00:08:56 +07:00
eafaeb226a [ceskatelevize] Lower priority for audio description sources (#12119) 2017-02-16 00:04:15 +07:00
de4d378c0c [ceskatelevize] Prefix format ids 2017-02-15 23:38:00 +07:00
099cfdb770 [devscripts/run_tests.sh] Change permission for script to 755 2017-02-16 00:28:31 +08:00
398dea3210 [test_YoutubeDL] Fix invalid escape sequences 2017-02-15 23:20:46 +07:00
db13c16ef8 [utils] Add support for quoted string literals in --match-filter (closes #8050, closes #12142, closes #12144) 2017-02-15 23:12:10 +07:00
1bd05345ea [amcnetworks] fix extraction(closes #12127) 2017-02-15 14:19:18 +01:00
3021cf83b7 [pinkbike] Fix uploader extraction (closes #12054) 2017-02-15 02:08:32 +07:00
04a741232f [onetpl] Add support for businessinsider.com.pl and plejada.pl 2017-02-15 01:23:55 +07:00
43a3d9edfc [onetpl] Add support for onet.pl (closes #10507) 2017-02-15 01:14:06 +07:00
d31aa74fdb [onetmvp] Add shortcut extractor 2017-02-15 00:58:18 +07:00
6092ccd058 [vodpl] Make more robust and add another test (closes #12122) 2017-02-15 00:52:31 +07:00
22ce9ad2bd [vod.pl] Add new extractor 2017-02-15 00:48:08 +07:00
9a372f14b4 [pornhub] Extract video URL from tv platform site (#12007, #12129) 2017-02-14 23:52:41 +07:00
5cb2d36c82 [ceskatelevize] Extract DASH formats (closes #12119, closes #12133) 2017-02-14 22:57:38 +07:00
fcca0d53a8 [ceskatelevize] Quick fix to revert to using old HLS-based playlist
This fixes recent changes in iVysilani. Proper patch should migrate to
MPEG-DASH version, which is now the default.
2017-02-14 22:25:37 +07:00
58a65ba852 release 2017.02.14 2017-02-14 01:09:18 +07:00
cedf08ff54 [ChangeLog] Actualize 2017-02-14 01:07:35 +07:00
50de3dbad3 [zdf] Fix extraction (closes #12117) 2017-02-14 01:00:06 +07:00
085f169ffe [xtube] Fix extraction for both kinds of video id (closes #12088) 2017-02-13 23:44:43 +07:00
f6d6ca1db3 [xtube] Improve title extraction 2017-02-13 23:34:14 +07:00
6e5956e6ba [lemonde] Fallback delegate extraction to generic extractor (closes #12115, closes #12116) 2017-02-13 23:17:48 +07:00
50fd3c2c69 Merge branch 'master' of github.com:rg3/youtube-dl 2017-02-13 22:58:50 +07:00
89c6691f9d [bellmedia] accept longer video id(closes #12114) 2017-02-13 15:08:48 +01:00
454e5cdb17 [limelight] add support referer protected videos 2017-02-13 14:29:05 +01:00
1de9f78e71 [travis] Separate builds for core and download 2017-02-13 18:56:05 +08:00
9dad941853 [disney] improve extraction
- add support for more urls
- detect expired videos
- skip Adobe Flash Access protected videos

closes #4975
closes #11000
closes #11882
closes #11936
2017-02-13 11:43:20 +01:00
1e2c3f61fc [travis] Separate builds for core and download 2017-02-13 17:36:13 +07:00
0dac7cbb09 [hotstar] improve extraction(closes #12096)
- extract all qualities
- detect drm protected videos
- extract more metadata
2017-02-12 17:35:24 +01:00
f8514630db [einthusan] Fix extraction (closes #11416)
The old test URLs are no longer valid, so I replace them with the one
from #11416
2017-02-12 20:53:55 +08:00
459818e280 [aenetworks] Add support for lifetimemovieclub.com 2017-02-12 20:18:11 +08:00
6310acf512 [youtube] Fix parsing codecs (closes #12091) 2017-02-12 18:09:53 +07:00
8d38dafbbf ChangeLog: update after #12085 2017-02-12 00:45:37 +08:00
f3915452de Merge pull request #12085 from wiiaboo/python2
utils.py: Workaround TypeError with Python 2.7.13 in Windows
2017-02-12 00:42:43 +08:00
2f49bcd690 utils.py: Workaround TypeError with Python 2.7.13 in Windows
Fixes #11540

Tested with Windows Python 2.7.12 and 2.7.13.
2017-02-11 14:51:28 +00:00
68c22c4c15 [iqiyi] Update _TESTS 2017-02-11 22:27:45 +08:00
9b92a5917b release 2017.02.11 2017-02-11 03:24:00 +07:00
3e2274c8b7 [ChangeLog] Actualize 2017-02-11 17:08:22 +07:00
3d7e3aaa0e [pluralsight:course] Fix extraction (closes #12075) 2017-02-11 17:00:52 +07:00
624c4b92ff [facebook] Add coding cookie 2017-02-11 16:18:45 +07:00
2af12ad9d2 Introduce get_elements_by_class and get_elements_by_attribute utility functions 2017-02-11 17:16:54 +08:00
97eb9bd2ac [bbc] extract m3u8 formats with 320k audio 2017-02-10 19:46:15 +01:00
71cdd75628 [facebook] Relax video id matching (closes #11017, closes #12055, closes #12056) 2017-02-11 01:05:22 +07:00
c7d6f614f3 [corus] Add new extractor(closes #12060)(#9164) 2017-02-10 17:00:09 +01:00
08a00eef79 [extractor/common] skip m3u8 manifests protected with Adobe Flash Access 2017-02-10 17:00:09 +01:00
9dd5408c99 [pluralsight] Detect blocked account error message (#12070) 2017-02-10 22:48:11 +07:00
9510709575 [bloomberg] Add another video id regex (closes #12062) 2017-02-10 22:16:20 +07:00
5abcca9060 [sixplay] use raw string for regex 2017-02-10 09:34:59 +01:00
e01bfc19c3 [extractor/commonmistakes] Restrict _VALID_URL (closes #12050) 2017-02-10 09:39:24 +07:00
4d32b63851 [tvplayer] Add new extractor 2017-02-09 23:09:21 +01:00
55d4de2283 release 2017.02.10 2017-02-10 01:27:33 +07:00
61ee556aea [ChangeLog] Actualize 2017-02-10 01:26:00 +07:00
ff24261ba0 [kaltura] Add explicit port to regexes
They should not match e.g. cdnapi.kaltura.computernetworks.com/...
2017-02-10 01:24:14 +07:00
fbc6dc525e [xtube] Fix shortcuts 2017-02-10 01:06:23 +07:00
9150d1eb69 [xtube] Fix extraction (closes #12023) 2017-02-10 01:03:35 +07:00
b7f9843bec [pornhub] Simplify (closes #12018) 2017-02-10 00:57:44 +07:00
e64b0fca14 [pornhub] Fix extraction (closes #12007) 2017-02-10 00:56:12 +07:00
78ef214d2d [facebook] Improve JS data regex (closes #12042) 2017-02-09 23:42:40 +07:00
be670b8e8f [external:ffmpeg] do not assume that ffmpeg unknown version format is new 2017-02-09 17:36:59 +01:00
37084f6641 [kaltura] improve embed partner id extraction(fixes #12041) 2017-02-09 16:24:54 +01:00
b04975733c [sprout] Add new extractor 2017-02-09 09:13:29 +01:00
c8b8fb0a99 [sixplay] improve extraction
- skip drm protected formats
- extract more and better formats
- skip duplicate asset urls
2017-02-08 22:56:10 +01:00
8298018273 [scrippsnetworks:watch] Add new extractor(closes #10765) 2017-02-08 20:44:23 +01:00
ae8d5a5c59 [go] add support for adobe pass auth(closes #11468)(closes #10831) 2017-02-08 18:57:07 +01:00
b9c9cb5f79 [6play] Fix extraction (closes #12011) 2017-02-08 23:15:39 +07:00
fdf9b959bc [nbc] add support adobe pass auth(closes #12006) 2017-02-08 16:23:42 +01:00
013877298d release 2017.02.07 2017-02-07 02:04:50 +07:00
c87f95f991 [ChangeLog] Actualize 2017-02-07 01:58:57 +07:00
f28aeff264 [pornhub] Fix extraction (closes #11997) 2017-02-07 01:52:59 +07:00
242a14a1f6 [extractor/common] Fix audio only with audio group in m3u8 (closes #11995) 2017-02-07 00:22:16 +07:00
d5d904ff7d [canalplus] Add support for cstar.fr (#11990) 2017-02-06 23:53:42 +07:00
5620f840f6 [extractor/generic] Add test for #11993 and more metadata for rtmp 2017-02-06 23:31:58 +07:00
b7a8c1bcfa [extractor/generic] Improve rtmp support (closes #11993) 2017-02-06 23:23:40 +07:00
7097bffba6 [downloader/fragment] Respect --no-part 2017-02-06 23:07:59 +07:00
2aec7256ae [extractor/common] Speed-up media tags regex (closes #11979) 2017-02-06 00:20:30 +07:00
815482d4eb Credit @motophil for gaskrank.py (#11685) 2017-02-06 00:38:22 +08:00
9c14fe9681 [gaskrank] Minor change and update ChangeLog after #11685 2017-02-06 00:25:28 +08:00
e705755739 [gaskrank] Add new extractor (#11685)
* [gaskrank] Add new extractor

* [gaskrank] Add new extractor - fixes as requested

* [gaskrank] Add new extractor - style fix

* [Gaskrank] Add new extractor - requested fixes

* [Gaskrank] Add new extractor - fix md5 checksum

* [gaskrank] Add new extractor - more requested fixes

* [Gaskrank] Add new extractor - fixed all but one quantified code issues

* [Gaskrank] add new extractor - more fields extracted, added second test

* [Gaskrank] Add new extractor - requested fixes.

* [Gaskrank] Add new extractor - requested changes.

* [Gaskrank] Add new extractor - final(?) fixes.
2017-02-06 00:19:37 +08:00
019f4c0371 [bandcamp] Fix extraction for incomplete albums
Closes #11727
2017-02-05 22:47:04 +08:00
2ab2c0d1f5 [iwara] Add width (closes #11724)
The heuristic is from #11724
2017-02-05 22:30:13 +08:00
caf0f5f8b7 [iwara] Fix extraction (closes #11781) 2017-02-05 21:48:13 +08:00
e4e50f60b1 [googledrive] Fix extraction on Python 3.6
Since Python 3.6, invalid escape sequences are deprecated. It's likely
that there are invalid escape sequences somewhere on the webpage, so
instead of unescaping the whole webpage, just unescape the URL.

See https://bugs.python.org/issue27364. That change was designed for
string literals, while it affects the 'unicode_escape' encoding as well.
The code path is:

str.decode('unicode_escape')
    codecs.unicode_escape_decode()
        PyUnicode_DecodeUnicodeEscape()
2017-02-05 21:41:08 +08:00
6ef3e65a7b [videopress] Add extractor 2017-02-05 13:37:27 +07:00
6fd138bed8 [sportbox] PEP 8 2017-02-05 13:36:52 +07:00
49bd8d5e2e [travis] Add python 3.6 2017-02-05 02:41:22 +07:00
3d2c2752c5 [afreecatv] extract rtmp formats 2017-02-04 18:18:28 +01:00
a713a86755 release 2017.02.04.1 2017-02-04 23:26:39 +07:00
7bccd5fc8a [ChangeLog] Actualize 2017-02-04 23:23:38 +07:00
3144eccf55 [ChangeLog] Actualize 2017-02-04 23:22:28 +07:00
9db8f6c540 [twitch:stream] Improve _VALID_URL (closes #11971) 2017-02-04 23:21:07 +07:00
8e4041cf3f [radiocanada] fix extraction for toutv rtmp formats 2017-02-04 17:05:35 +01:00
31487eb974 release 2017.02.04 2017-02-04 22:57:48 +07:00
c2521c1ac6 [Piksel] Add another app token regex 2017-02-04 23:23:14 +08:00
643dc0fcfe [vk] Catch author blocked error message
Example link (video in blocked group):
https://vk.com/search?c%5Bq%5D=%D0%9F%D1%80%D1%8B%D0%B6%D0%BE%D0%BA%20c%20%D0%BA%D1%80%D0%B0%D0%BD%D0%B0%20%D0%B2%20%D1%81%D1%82%D0%B8%D0%BB%D0%B5%20%D0%A7%D0%B5%D0%BB%D0%BE%D0%B2%D0%B5%D0%BA%D0%B0-%D0%BF%D0%B0%D1%83%D0%BA%D0%B0&c%5Bsection%5D=video&c%5Bsort%5D=2&z=video-10639516_456240611
2017-02-04 22:21:09 +07:00
36fce54816 [turner] fix downloading of secure hls formats using ffmpeg(closes #11358)(closes #11373)(closes #11800) 2017-02-04 15:23:46 +01:00
2c15db829c [drtv] add support for live and radio sections(closes #1827)(closes #3427) 2017-02-04 08:38:28 +01:00
f65dba7cdb [myspace] fix extraction and extract hls and http formats 2017-02-03 22:25:19 +01:00
605fd6392f [youtube] add format info for itag 325 and 328 2017-02-03 17:59:48 +01:00
f962790ee5 [vine] Fix extraction (closes #11955) 2017-02-03 21:56:48 +07:00
b7cc5f078e [extractors] Remove remnants of sportbox extractor (#11954) 2017-02-03 21:56:10 +07:00
f7a10d8cd6 [sportbox] Remove extractor (closes #11954)
Covered by generic extractor
2017-02-03 21:25:44 +07:00
daac118bf4 [ChangeLog] Update after #11901 2017-02-03 18:56:40 +08:00
8939f784d9 Merge pull request #11901 from ThomasChr/randonplaylistorder
New parameter --playlist-random to randomize playlist download order. Fixes #11889
2017-02-03 18:53:14 +08:00
df0588a31f Merge branch 'fstirlitz-filmon' 2017-02-03 10:15:52 +01:00
4ce3407d08 [filmon] improve extraction 2017-02-03 10:15:03 +01:00
d7f9242e30 [ChangeLog] Update after #11565 2017-02-03 12:13:24 +08:00
45024183ae [infoq] Add audio only format if available (#11565)
* [infoq] Add audio only format if available

Refactor cookie code into a function.
Renamed formats to http_video, http_audio, rtmp_video
Renamed extract functions to video instead of videos as they return
one or no video.

* [infoq] Rename to _extract_cookies as it more than one

* [infoq] Remove redundant determine_ext

* [infoq] Add comment about hardcoded URL

* [infoq] Use _hidden_inputs instead of messy regex

* [infoq] Probe if audio URL is valid

Make it possible to pass headers to _is_valid_url

* [infoq] Add audio only test
2017-02-03 12:10:13 +08:00
33da98f493 [douyutv] Improve room id regex
http://www.douyu.com/t/lpl  source get extra '\' with "room_id\" (from js coding)
2017-02-03 03:26:41 +07:00
4195096ea8 [utils] Improve comments processing in js_to_json (closes #11947) 2017-02-03 03:04:33 +07:00
0bbcc8a10a [iprima] Fix extraction (closes #11920, closes #11896) 2017-02-03 03:04:33 +07:00
b3ee552e4b [utils] Handle single-line comments in js_to_json 2017-02-03 03:04:33 +07:00
a22b2fd19b [youtube] Fix ytsearch* when cookies are provided
Closes #11924

The API with `page` is no longer used in browsers, and YouTube always
returns {'reload': 'now'} when cookies are provided.

See http://youtube.github.io/spfjs/documentation/start/ for how SPF
works. Basically appending static link with a `spf` parameter yields the
corresponding dynamic link.
2017-02-03 01:28:24 +08:00
c54c01f82d [go] Relax video id regex (closes #11937) 2017-02-02 23:04:46 +07:00
5a116e1302 [facebook] Fix title extraction (closes #11941) 2017-02-02 22:45:18 +07:00
a685751051 [youtube:playlist] Recognize TL playlists (closes #11945) 2017-02-02 22:01:11 +07:00
bd8f48c78b [bilibili] Support new Bangumi URLs (closes #11845)
To reduce complexity, I don't support old Bangumi URLs directly via
_VALID_URL. Instead, I choose to let it go to generic redirection. An
example can be found in #10190:

http://bangumi.bilibili.com/anime/v/40062
2017-02-02 21:51:31 +08:00
81aeafeb44 [cbc:watch] extract audio codec for audion only formats(fixes #11893) 2017-02-02 08:07:28 +01:00
8bdc149441 [downloader/external:ffmpeg] minimize the use of aac_adtstoasc filter 2017-02-02 08:07:28 +01:00
020c5df52d [elpais] Fix extraction for some URLs (closes #11765) 2017-02-01 23:48:34 +01:00
da162c1135 [compat] add compat_etree_register_namespace to __all__ list 2017-02-01 20:15:59 +01:00
5069594993 release 2017.02.01 2017-02-01 03:20:09 +07:00
b996b88092 [ChangeLog] Actualize 2017-02-01 23:29:59 +07:00
b83ef507b4 [facebook] Fix extraction (closes #11926) 2017-02-01 23:19:20 +07:00
000f207944 [prosiebensat1] Update ChangeLog 2017-02-01 23:16:35 +08:00
fe5aa197b5 [prosiebensat1] PEP8 and update _TESTS 2017-02-01 23:13:45 +08:00
7882f1115e Added new Regex for prosiebensat1 Extractor Description. Fixes #11810 (#11929)
* Added new Regex for prosiebensat1 Extractor Description. Fixes #11810

* Using _og_search_description() as a Fallback for Description-Regex

* Using _og_search_description() as a Fallback for Description-Regex - Second try

* Also added fallback regex

* Using _og_search_description() as a Fallback for Description-Regex - Third try

* removed fatal=False from search for description regex. default=None should be preferred only

* Using fatal=false for _og_search_description

* Revert "Using fatal=false for _og_search_description"

This reverts commit 2b7e123f9d.

* Deleted default=None Parameter for _og_search_property
2017-02-01 23:00:41 +08:00
2b2d5d319b [crunchyroll] Remove ScaledBorderAndShadow settings
See https://github.com/rg3/youtube-dl/pull/9028, especially @lachs0r's
comments for the reason behind this change
2017-02-01 16:39:32 +08:00
26c0f09935 [vimeo] PEP 8 2017-02-01 02:15:52 +07:00
c15cd29640 [vimeo] Extract upload timestamp 2017-02-01 00:58:02 +07:00
c38a67bcd5 [vimeo] Extract license (closes #11880) 2017-02-01 00:49:28 +07:00
363245ad94 Credit @oddstr13 for nrk:series (#11571) 2017-02-01 00:30:19 +07:00
7c5329e6f4 [nrk] Improve extraction and update tests (closes #11571) 2017-02-01 00:29:29 +07:00
8fd65faece [NRKTV] Added NRKTVSeriesIE
[NRKTV] Added season and episode number to metadata.

[NRKTV] Added category to metadata.

[NRKTV] Added tests to NRKTVSeries.

[NRKTV] Fixed whitespace issues (flake8).
2017-01-31 23:10:40 +07:00
d7e215b42d release 2017.01.31 2017-01-31 22:24:45 +07:00
3a528ffd89 [ChangeLog] Actualize 2017-01-31 22:22:42 +07:00
3c90cc8b6f [youtube] Fix extraction for domainless player URLs
Closes #11890
Closes #11891
Closes #11892
Closes #11894
Closes #11895
Closes #11897
Closes #11900
Closes #11903
Closes #11904
Closes #11906
Closes #11907
Closes #11909
Closes #11913
Closes #11914
Closes #11915
Closes #11916
Closes #11917
Closes #11918
Closes #11919
2017-01-31 22:22:37 +07:00
ae9a173b64 [vimeo] extract both mixed and separated dash formats 2017-01-31 14:47:56 +01:00
75822ca790 New parameter --playlist-random to randomize playlist download order. Fixes #11889 2017-01-31 10:03:31 +01:00
dadb836139 [ruutu] extract dash formats 2017-01-30 09:32:31 +01:00
4719419951 [itv] fix extraction in python 2.6 2017-01-29 16:04:15 +01:00
c2d9c25f81 [compat] add compat_etree_register_namespace 2017-01-29 16:03:39 +01:00
4d2fdb07c4 release 2017.01.29 2017-01-29 13:21:42 +07:00
fe323a4800 [ChangeLog] Actualize 2017-01-29 21:21:26 +07:00
f13da8af28 [azmedien:playlist] Add support for topic and themen playlists 2017-01-29 20:40:19 +07:00
e228616c6e [extractor/common] Fix initialization template (closes #11605, closes #11825) 2017-01-29 06:57:39 +07:00
c58c2d63cb [extractor/common] Document forgotten fragment base and path interfaces 2017-01-29 05:57:53 +07:00
d04621daf4 [extractor/common] Fix duration per dash segment (closes #11868) 2017-01-29 05:36:53 +07:00
76aaf1faae Credit @BagiraHun for videa (#11133) 2017-01-29 03:44:11 +07:00
56fc078da8 [npo] Update subtitles url
NPO websites changed the domain they used for subtitles, from
e.omroep.nl to tt888.omroep.nl.
2017-01-29 02:29:39 +07:00
0842b8241d Credit @fast90 for config location (#10648) 2017-01-29 01:03:59 +07:00
59c307891a Credit @RPing for cntv (#8541) 2017-01-29 01:02:28 +07:00
4d07b748c2 Credit @bastik for zdf fix (#11063) 2017-01-29 01:01:39 +07:00
f5169501d2 Credit @sudovijay for openload fix (#11646) 2017-01-29 01:00:17 +07:00
186f4abe93 Credit @goggle for 20min (#11683) and azmedien (#11805) 2017-01-29 00:59:17 +07:00
34cea6137e Credit @einstein95 for pornflip (#11795) and chaturbate fix (#11797) 2017-01-29 00:57:15 +07:00
ffcfb7e3e0 Credit @costypetrisor for autonumber start (#9362) 2017-01-29 00:54:31 +07:00
c0af11abee Credit @AVerwer for showroomlive (#11458) 2017-01-29 00:52:23 +07:00
1a241a2d02 [options] Refactor autonumber options and add validation (closes #727, closes #2702, closes #9362) 2017-01-29 00:00:45 +07:00
acbb2374bc added --autonumber-start NUMBER as a command line option to be able to offset the index at which autonumber formats filenames 2017-01-29 00:00:45 +07:00
4edeac5bfa [itv] fix subtitle extraction 2017-01-28 17:28:18 +01:00
f592ff9868 [itv] extract subtitles 2017-01-28 17:26:14 +01:00
24ee6b9721 [options] Remove experimental mark from some options 2017-01-28 22:41:04 +07:00
a71b8d3b3b [itv] Add new extractor(closes #9240) 2017-01-28 15:52:50 +01:00
732fb3f8be [options] Move --abort-on-unavailable-fragment to download section 2017-01-28 21:06:22 +07:00
008f247077 [mtv81] Add extractor (closes #7619) 2017-01-28 20:29:22 +07:00
661cc229d2 [vlive:channel] Improve 2017-01-28 19:08:01 +07:00
b92d3c5343 [vlive] Add support for channels 2017-01-28 18:50:38 +07:00
ab6f6aee78 [kaltura] Add fallback for fileExt 2017-01-28 18:27:42 +07:00
26e40542dd [kaltura] Improve uploader_id extraction 2017-01-28 17:50:56 +07:00
99a0baf370 [konserthusetplay] Add support for rspoplay.se 2017-01-28 01:58:23 +07:00
d41ed6d243 release 2017.01.28 2017-01-28 00:33:55 +07:00
815d2a36d8 [ChangeLog] Actualize 2017-01-28 00:03:21 +07:00
e0b6e50ccd [crunchyroll] Improve series and season metadata extraction (closes #11832) 2017-01-27 23:55:55 +07:00
3a194cb4ec [soundcloud] Improve formats extraction and extract audio bitrate 2017-01-27 23:16:30 +07:00
9b73471801 [soundcloud] Extract hls formats 2017-01-27 23:08:32 +07:00
489ffc1182 [soundcloud] Fix track URL extraction (closes #11852) 2017-01-27 22:55:42 +07:00
0b23c222ba [twitch:vod] Expand _VALID_URL (closes #11846) 2017-01-27 21:33:05 +07:00
b51a4ebed4 [aenetworks] fix season episodes extraction(fixes #11669) 2017-01-26 19:15:43 +01:00
9463637887 [tva] Add new extractor(closes #11842) 2017-01-26 18:37:22 +01:00
3cbecdd111 [jamendo] Improve and extract more metadata (closes #11836) 2017-01-26 23:26:18 +07:00
15846398ca [utils] Improve parse_duration 2017-01-26 23:23:08 +07:00
c19ef77c31 [jamendo] Extract full title 2017-01-26 22:53:16 +07:00
b3277115a1 [disney] Add new extractor(closes #7409)(closes #11801)(#4975)(#11000) 2017-01-26 16:16:14 +01:00
9bccdc7004 [vevo] remove request to old api and catch apiv2 errors 2017-01-26 16:16:14 +01:00
cf0cabbe50 [cmt,mtv,southpark] Add support for episode URLs (closes #11837) 2017-01-26 21:49:34 +07:00
556dbe7fe3 [youtube] Add fallback for duration extraction (closes #11841) 2017-01-26 21:43:14 +07:00
2417d41535 release 2017.01.25 2017-01-25 23:36:03 +07:00
2c302cf66b [ChangeLog] Actualize 2017-01-25 23:33:46 +07:00
c1fa3f4672 [openload] Fallback video extension to mp4 2017-01-25 23:28:45 +07:00
17f8deeb48 [extractor/generic] Add support for openload embeds (closes #11536, closes #11812) 2017-01-25 23:27:22 +07:00
b8a03b6660 [srgssr] fix rts video extraction(closes #11831) 2017-01-25 07:39:41 +01:00
c60089c022 [afreecatv:global] Add new extractor(closes #11807) 2017-01-25 07:39:41 +01:00
af59bddc4e [konserthusetplay] Extract subtitles (#11823) 2017-01-24 23:03:09 +07:00
23b35a634e [crackle] improve extraction
- extract vtt subtitles
- extract multiple resolutions for thumbnails
- pass geo verification proxy headers
- add support for mobile urls
2017-01-24 16:57:22 +01:00
74af9c700d [konserthusetplay] Add support for hls formats (closes #11823) 2017-01-24 22:55:49 +07:00
d61aa5eb37 [vimeo:review] Fix config URL extraction (closes #11821) 2017-01-24 22:46:40 +07:00
c3a65c3de0 release 2017.01.24 2017-01-24 02:58:37 +07:00
ee4c091ce5 [ChangeLog] Actualize 2017-01-24 02:56:19 +07:00
b494d6856c [pluralsight] Fix extraction (closes #11820) 2017-01-24 02:51:45 +07:00
bc35ed3fb6 [nextmedia] Add support for NextTV (壹電視) 2017-01-23 23:38:31 +08:00
0c1c6f4b9f [utils] Add another date format seen in NextTV 2017-01-23 23:37:32 +08:00
6d119c2a6b [24video] Fix extraction (closes #11811) 2017-01-23 03:50:39 +07:00
4201ba13e6 [youtube:playlist] Fix nonexistent/private playlist detection and skip private tests 2017-01-23 02:49:56 +07:00
8bc0800d7c [youtube:playlist] Fix nonexistent/private playlist detection (closes #11604) 2017-01-23 02:35:38 +07:00
a089545e03 [azmedien:show] Improve _VALID_URL 2017-01-23 02:30:29 +07:00
30dda24de3 [chirbit] Extract uploader 2017-01-23 02:27:38 +07:00
9d5b29c881 release 2017.01.22 2017-01-22 18:59:04 +07:00
6c031a35f3 [ChangeLog] Actualize 2017-01-22 18:57:15 +07:00
271808b6b2 [pornflip] Improve and extract dash formats (closes #11795) 2017-01-22 03:43:27 +07:00
8d1fbe0cb2 [pornflip] Add extractor (closes #11556) 2017-01-22 03:41:59 +07:00
a243abb80d [chaturbate] Improve (closes #11797) 2017-01-22 03:02:48 +07:00
42697bab3c [chaturbate] Fix extraction 2017-01-22 02:58:40 +07:00
94629e537f [azmedien] Improve (closes #11784) 2017-01-22 02:17:39 +07:00
e84495cd8d [azmedien] Add extractor (closes #11785) 2017-01-22 02:17:39 +07:00
7c20b7484c [nextmedia] Support redirected URLs 2017-01-22 02:06:34 +08:00
04a3d4d234 [vimeo:channel] Extract videos' titles for playlist entries 2017-01-21 23:37:44 +07:00
12afdc2ad6 [youtube] Extract episode metadata (closes #9695, closes #11774) 2017-01-21 18:10:32 +07:00
f4ec8dce48 Update README.md (#11787)
Add audio format argument dependency warning
2017-01-21 00:25:04 +08:00
f3c21cb7a7 [cspan] Fix _TESTS 2017-01-20 22:27:13 +08:00
972efe60c3 [generic] Remove a dead test
The web page does not contain a video anymore

Ref: #2694, #2696
2017-01-20 22:27:13 +08:00
4447fb2332 [cspan] Support Ustream embedded videos
Closes #11547
2017-01-20 22:27:13 +08:00
d77ac73790 [ustream] Add UstreamIE._extract_url()
Ref: #11547
2017-01-20 22:27:13 +08:00
1fe84be0f3 [1tv] Add support for hls (closes #11786) 2017-01-20 00:47:04 +07:00
1076858f76 Merge pull request #11778 from h4ck3rm1k3/master
Fix typos
2017-01-19 19:32:29 +08:00
cccd70a275 untie 2017-01-19 04:18:13 -05:00
eb3f008c9e [uol] Fix extraction (closes #11770) 2017-01-19 04:49:31 +07:00
f1e70fc2ff [mtv] Relax triforce feed regex (closes #11766) 2017-01-18 23:34:11 +07:00
1560baacc6 release 2017.01.18 2017-01-18 23:10:00 +07:00
460f61fac4 [ChangeLog] Actualize 2017-01-18 23:06:46 +07:00
baa3e1845b [bilibili] fix extraction(closes #11077) 2017-01-18 17:00:15 +01:00
aaf2b7c57a [canalplus] Add fallback for video id (closes #11764) 2017-01-18 22:20:11 +07:00
b687c85eab [extractor/generic] Add support for 20 minuten embeds (closes #11683, closes #11751) 2017-01-18 22:17:47 +07:00
538b17a09c [20min] Improve 2017-01-18 22:17:41 +07:00
4e44598547 [20min] Fix extraction 2017-01-18 22:17:34 +07:00
136078966b [imdb] Extend _VALID_URL (closes #11744) 2017-01-17 23:14:07 +07:00
8a5f0a6357 [naver] Update tests for #11743 2017-01-17 21:19:57 +07:00
c0bd51c090 [naver] Support tv.naver.com links 2017-01-17 21:15:50 +07:00
c1c2fe2045 release 2017.01.16 2017-01-16 23:44:04 +07:00
ddd53c392e [ChangeLog] Actualize 2017-01-16 23:42:04 +07:00
79fc8496c6 [xiami] Improve extraction (closes #11699)
* Relax _VALID_URLs
* Improve track metadata extraction
2017-01-16 23:31:50 +07:00
0ce8c66fb0 [options] Include custom conf in final argv (closes #11741) 2017-01-16 22:07:12 +07:00
906420cae3 [limelight] Improve and make more robust (closes #11737)
+ Add support for direct http for videos hosted on video.llnw.net
* Check handmade http URLs
2017-01-16 21:54:47 +07:00
16e2c8f771 [brightcove] Recognize another player ID
Closes #11688
2017-01-16 00:06:52 +08:00
dcae7b3fdc [niconico] Allow login via cookies
Some codes are borrowed from #7968, which is by @jlhg

Closes #7968
2017-01-15 22:51:54 +08:00
8e4988f1a2 [niconico] Remove codes for downloading anonymously
Apparently Niconico now blocks playing without an account

Closes #11170
2017-01-15 22:10:57 +08:00
a7acf868a5 [yourupload] Fix extraction (closes #11601) 2017-01-15 10:34:39 +07:00
6f0be93747 [YoutubeDL] Improve protocol auto determining (closes #11720) 2017-01-15 06:09:32 +07:00
af62de104f [beam:live] Improve and simplify (#10702, closes #11596) 2017-01-15 06:07:35 +07:00
cd55c6ccd7 [beam:live] Add extractor 2017-01-15 06:06:10 +07:00
621a2800ca [vevo] Improve geo restriction detection 2017-01-15 04:42:05 +07:00
b80e2ebc8d [dramafever] Add support for URLs with language code (#11714) 2017-01-14 18:27:22 +07:00
99d537a5e0 [ooyala] fix typo 2017-01-14 07:12:50 +01:00
8854f3fe78 [README.md] Clarify newline format in cookies section (closes #11709) 2017-01-14 08:48:26 +07:00
abe8cb763f [cbc] Improve playlist support (closes #11704) 2017-01-14 08:30:00 +07:00
5d4c7daa49 release 2017.01.14 2017-01-14 07:31:07 +07:00
0b94510cd0 [ChangeLog] Actualize 2017-01-14 07:30:32 +07:00
4f66c16f33 [brightcove:legacy] Fix misplaced backslash in a regexp 2017-01-14 06:26:11 +07:00
e54fc0524e [cmt] Add support for video-clips 2017-01-14 06:23:24 +07:00
adf063dad1 [mtv,cc,cmt,spike] Improve and refactor
- Eliminate _transform_rtmp_url
* Generalize triforce mgid extraction
+ [cmt] Add support for full-episodes (closes #11623)
2017-01-14 06:18:38 +07:00
5e8eebb600 [mitele] extract dash formats 2017-01-13 23:06:59 +01:00
9837cb7507 [ooyala] add support for videos with embedToken(#11684) 2017-01-13 23:06:59 +01:00
fb6a59205e [mixcloud] Fix extraction (closes #11674) 2017-01-13 23:56:16 +07:00
06e9363b7a [openload] Fix extraction (closes #10408)
Just a minor fix for openload
2017-01-13 23:40:19 +07:00
1f393a3241 [tv4] improve extraction(closes #11698)
- remove check for requires_subscription
- extract more formats
- extract subtitles
2017-01-13 10:21:37 +01:00
c4251b9aaa [common] add possibility to customize akamai manifest host 2017-01-13 10:21:36 +01:00
3a407e707a [freesound] Improve and remove unrelated metadata (closes #11608) 2017-01-12 23:03:53 +07:00
cb655f34fb [utils] Add more date formats 2017-01-12 22:39:45 +07:00
ed06da4e7b [freesound] Fix extraction and extended (closes #11602) 2017-01-12 22:35:14 +07:00
365d136b7c [vimeo] Fix tests 2017-01-11 22:57:08 +07:00
1fd0fc42bd [vimeo:ondemand] Fix test (closes #11651) 2017-01-11 22:51:03 +07:00
10cd2003b4 [nick] Add support for beta.nick.com (closes #11655) 2017-01-10 22:32:34 +07:00
cdd11c0540 [mtv] Use native hls by default 2017-01-10 22:31:20 +07:00
67fc365b86 [mtv,cc] Use hls by default (closes #11641) 2017-01-10 22:30:47 +07:00
20faad74b6 [mtv] Fix non-hls extraction
method attribute may not be present
2017-01-10 22:27:23 +07:00
2032d935d1 [mtv] Add default value for use_hls
These methods are used across codebase with old number of arguments
2017-01-10 22:25:33 +07:00
31ea2ad89d release 2017.01.10 2017-01-10 21:29:20 +07:00
2184d44361 [ChangeLog] Actualize 2017-01-10 21:27:17 +07:00
d1aeacd9bf [youtube] Fix extraction (closes #11663, #11664) 2017-01-10 21:25:29 +07:00
366b759a60 [inc] Improve (closes #11647) 2017-01-09 23:08:59 +07:00
7f0bdc7a31 [inc] Add extractor 2017-01-09 22:57:14 +07:00
022a5d663b [youtube] Add test for itag 212 (#11575) 2017-01-09 22:30:46 +07:00
8409b3683c [youtube] Add itag 212
Seen on video with id 1t24XAntNCY
2017-01-09 22:29:03 +07:00
bfedb2cc5a small fix to Changelog format 2017-01-09 11:26:01 +01:00
8084951b7f [egghead:course] Add support for egghead.io course playlists
Individual egghead videos are already handled by the generic/Wistia extractors.
2017-01-09 11:24:40 +01:00
e7ea724cb9 release 2017.01.08 2017-01-08 20:58:43 +07:00
e60166020b [ChangeLog] Actualize 2017-01-08 20:56:38 +07:00
364131584b [hitrecord] Improve (closes #11626) 2017-01-08 20:17:18 +07:00
J
553c68bbd9 [hitrecord] Add extractor 2017-01-08 20:17:18 +07:00
827961b122 [videott] remove extractor 2017-01-07 14:47:36 +01:00
a5eefc492b [swrmediathek] skip tests correctly 2017-01-06 15:09:10 +01:00
a9cd1691b2 [swrmediathek] improve extraction 2017-01-06 15:06:08 +01:00
2365f94412 [sharesix] remove extractor 2017-01-06 13:56:58 +01:00
32b7c2a57e [aol] remove AolFeaturesIE 2017-01-06 12:10:47 +01:00
221ce32529 [break] merge BreakIE and ScreenJunkiesIE 2017-01-06 11:25:48 +01:00
e5dfdc8164 [sendtonews] improve info extraction 2017-01-06 11:23:43 +01:00
a814da3f62 [skynews] update test 2017-01-06 11:22:35 +01:00
b2727d0bee [3sat,phoenix] Fix extraction (closes #11619) 2017-01-06 17:13:53 +07:00
dbaf601646 [comedycentral/mtv] Add support for HLS videos (fixes #11600)
Currently, the HTTP files of the RTMP urls are not present for the The Daily Show.
Use HLS instead for now.
2017-01-05 22:36:07 +01:00
a9ee260217 [README.md] Mention --config-location in configuration section (#11615) 2017-01-06 02:45:45 +07:00
1219201143 [ChangeLog] Update after #11581
[ci skip]
2017-01-06 01:07:30 +08:00
ec85ded83c Fix "invalid escape sequences" error on Python 3.6 2017-01-06 00:58:56 +08:00
24d8a75982 [discoverygo] Fix JSON data parsing
HTMLParser, which is used by extract_attributes, already unescapes
attribute values with HTMLParser.unescape. They shouldn't be unescaped
again, to there may be parsing errors.

Ref: #11219, #11522
2017-01-05 18:50:34 +08:00
7232bb299b release 2017.01.05 2017-01-05 04:10:15 +07:00
2b12e34076 [ChangeLog] Actualize 2017-01-05 04:07:32 +07:00
fb47cb5b23 [zdf] Improve (closes #11055, closes #11063) 2017-01-05 04:05:27 +07:00
b6de53ea8a [zdf] Fix extraction 2017-01-05 04:04:53 +07:00
96d315c2be [pornhub:playlist] Improve extraction (closes #11594) 2017-01-04 05:32:18 +07:00
1911d77d28 [cctv] Add support for ncpa-classic.com (closes #11591) 2017-01-04 01:30:40 +07:00
027e231295 [tunein] Add support for embeds (closes #11579) 2017-01-03 01:45:59 +07:00
7a9e066972 [cctv] Relax some video id regexes 2017-01-03 01:13:02 +07:00
2021b650dd release 2017.01.02 2017-01-02 23:55:04 +07:00
b890caaf21 [ChangeLog] Actualize 2017-01-02 23:54:29 +07:00
3783a5ccba [cctv] Relax _VALID_URL 2017-01-02 23:18:44 +07:00
327caf661a [cctv] Do not fallback on video id extracted from URL 2017-01-02 23:00:37 +07:00
ce7ccb1caa [cctv] Improve and merge with cntv (closes #879, closes #6753, closes #8541) 2017-01-02 22:55:24 +07:00
295eac6165 [cntv] Add extractor 2017-01-02 22:55:19 +07:00
d546d4c8e0 Merge pull request #11578 from rpunkfu/patch-2
Update Readme: Set HOME properly in install command
2017-01-02 13:50:26 +08:00
eec45445a8 Update Readme: Set home in sudo pip install
Hi, it's not always a default behaviour, but when you use `sudo pip` you most likely may want to use `-H` flag to set HOME value to directory of the target user :)
2017-01-02 00:22:10 +01:00
7fc06b6a15 [README.md] Update link to available YoutubeDL options 2017-01-01 23:36:52 +07:00
966815e139 [nrktv:episodes] Add support for episodes (#11571) 2017-01-01 21:26:32 +07:00
e5e19379be [ISSUE_TEMPLATE_tmpl.md] Clarify example URLs constraints for site support request 2017-01-01 15:11:49 +07:00
1f766b6e7b [arkena] Add support for video.arkena.com (closes #11568) 2017-01-01 02:46:47 +07:00
dc48a35404 release 2016.12.31 2016-12-31 23:58:41 +07:00
1ea0b727c4 [ChangeLog] Actualize 2016-12-31 23:53:30 +07:00
b6ee45e9fa Improve custom config support (closes #10648) 2016-12-31 23:41:37 +07:00
e66dca5e4a Add option --config-location
A configfile can now be passed to youtube_dl.

undo changes

Raise parser error if file not found, change to user_conf

change metavar hand helptext for --configfile

Fix help for --configfile

Update help for --configfile

Numbering placeholder in configfile error msg

minor fix

Change option --configfile top --config-file

Fix -config-file error
2016-12-31 23:04:16 +07:00
3f1ce16876 [twitch:vod] Improve _VALID_URL (closes #11537) 2016-12-31 22:40:42 +07:00
9a0f999585 [twitch] Added support for player.twitch.tv URLs (closes #11535) 2016-12-31 22:32:49 +07:00
3540fe262f [README.md] Fix spelling and harmonize line length 2016-12-31 22:29:35 +07:00
e186a9ec03 [videa] Add support for videa embeds 2016-12-31 22:05:32 +07:00
69677f3ee2 [videa] Improve and simplify (closes #8181, closes #11133) 2016-12-31 22:05:32 +07:00
e746021577 [videa] Add extractor 2016-12-31 22:05:32 +07:00
490da94edf [devscripts/buildserver] Remove unreachable except block 2016-12-31 19:17:52 +07:00
424ed37ec4 [vk] Fix postlive videos extraction 2016-12-30 04:31:19 +07:00
9cdb0a338d [vk] Extract from playerParams (closes #11555) 2016-12-30 04:21:49 +07:00
6cf261d882 [freevideo] Remove extractor (closes #11515)
Handled by generic extractor
2016-12-30 00:32:23 +07:00
df086e74e2 [showroomlive] Improve (closes #11458) 2016-12-30 00:12:35 +07:00
963bd5ecfc [showroomlive] Add extractor 2016-12-29 23:17:00 +07:00
51378d359e [xhamster] Fix duration extraction (closes #11549) 2016-12-28 23:04:46 +07:00
b63005f5af [rtve:live] Fix extraction (closes #11529) 2016-12-25 04:02:29 +07:00
4606c34e19 [extractor/common] Allow non-lang in subtitles' keys
See 264e77c406
2016-12-25 01:50:50 +08:00
53a664edf4 [brightcove:legacy] Improve embeds detection (closes #11523) 2016-12-24 22:46:27 +07:00
264e77c406 [twitch] Add support for rechat messages (closes #11524) 2016-12-24 22:10:54 +07:00
d1cd7e0ed9 Credit @wader for #11521 2016-12-24 15:00:23 +01:00
846fd69bac [acast] Add test with multiple blings 2016-12-24 14:28:30 +01:00
12da830993 [acast] Fix broken audio URL and timestamp extraction
Before first bling was used now we look for the first bling with
type BlingAudio.

Before publishingDate was a ms unix timestamp now it is iso8601.
2016-12-24 14:28:30 +01:00
e7ac722d62 [README.md] Add missing protocols to format selection section 2016-12-23 22:01:22 +07:00
19f37ce4b1 [README.md] Fix typo 2016-12-23 09:25:39 +07:00
5e77c0b58e release 2016.12.22 2016-12-22 22:52:54 +07:00
ab3091feda [ChangeLog] Actualize 2016-12-22 22:51:51 +07:00
a07588369f [common] improve detection for video only formats and m3u8 manifest(fixes #11507) 2016-12-22 10:02:56 +01:00
f5a723a78a [theplatform] pass geo verification headers to smil request(closes #10146) 2016-12-21 20:59:03 +01:00
f120646f04 [viu] pass geo verification headers to auth request 2016-12-21 20:50:10 +01:00
9c5b5f2115 [rtl2] extract more formats and metadata 2016-12-21 18:46:25 +01:00
ae806db628 [vbox7] Skip malformed JSON-LD (closes #11501) 2016-12-21 22:39:05 +07:00
bfa1073e11 [uplynk] force downloading using hls native downloader(closes #11496) 2016-12-20 19:49:45 +01:00
e029c43bd4 [laola1] add support for another extraction scenario(closes #11460) 2016-12-20 18:22:57 +01:00
90352a8041 release 2016.12.20 2016-12-20 22:39:39 +07:00
1f6a79b0af [ChangeLog] Actualize 2016-12-20 22:37:06 +07:00
3d6761ba92 [vbox7] Fix extraction (closes #11494) 2016-12-20 21:53:51 +07:00
f59d1146c0 [uktvplay] Add new extractor(closes #11027) 2016-12-20 12:52:46 +01:00
b1c357975d [piksel] Add new extractor(closes #11246) 2016-12-20 12:35:03 +01:00
d8c507c9e2 [vimeo] fix extraction for hls formats and add support for dash formats(closes #11490) 2016-12-20 12:35:03 +01:00
7fe1592073 [common] fix dash codec information for mixed videos and fragment url construction(#11490) 2016-12-20 12:35:03 +01:00
8ab7e6c4cc [kaltura] Improve widget ID extraction (closes #11480) 2016-12-20 18:45:52 +08:00
c80db5d398 [nrktv:direkte] Add support for live streams (#11488) 2016-12-19 23:47:45 +07:00
5aaf012a4e [pbs] fix extraction for geo restricted videos(#7095) 2016-12-19 16:27:12 +01:00
954529c10f [brightcove:new] skip widevine classic videos 2016-12-18 21:39:59 +01:00
ed7b333fbf [viu] extract supported hls manifest 2016-12-18 18:24:01 +01:00
723103151e [viu] improve extraction(closes #10607)(closes #11329) 2016-12-18 17:20:53 +01:00
e7b6caef24 [viu] New extractor for viu.com 2016-12-18 17:20:53 +01:00
ec79b1de1c Revert "Credit @pyx for meipai (#10718)"
This reverts commit d5e623aaa1.
2016-12-18 20:56:21 +07:00
f73d7d5074 release 2016.12.18 2016-12-18 19:50:33 +07:00
52a1d48d9f [ChangeLog] Actualize 2016-12-18 19:48:59 +07:00
d5e623aaa1 Credit @pyx for meipai (#10718) 2016-12-18 19:46:57 +07:00
199a47abba [ccma] Add new extractor(closes #11359) 2016-12-18 10:49:10 +01:00
b42a0bf360 [laola1tv] add support embed urls and improve extraction(#11460) 2016-12-17 21:48:45 +01:00
6e416b210c [nbc] fix extraction for msnbc videos(fixes #11466) 2016-12-17 18:11:13 +01:00
04bf59ff64 [extractors] Add missing twitch imports 2016-12-17 23:03:50 +07:00
87a449c1ed [extractor/common] Recognize DASH formats in html5 media entries 2016-12-17 23:03:13 +07:00
93753aad20 [twitch] Adapt to new videos pages schema (closes #11469) 2016-12-17 20:20:23 +07:00
2786818c33 [meipai] Fix regular videos extraction and improve (closes #10718) 2016-12-17 19:42:34 +07:00
9b785768ac [meipai] Add extractor 2016-12-17 19:41:35 +07:00
47c914f995 [ondemandkorea] Fix extraction (closes #10772) 2016-12-17 18:50:12 +07:00
732d116aa7 [jwplatform] Improve duration extraction 2016-12-17 18:50:07 +07:00
a495840d3b [jwplatform] Improve subtitles extraction 2016-12-17 18:50:00 +07:00
b0c65c677f [utils] Improve urljoin 2016-12-17 18:49:55 +07:00
594601f545 [ondemandkorea] Add extractor 2016-12-17 18:49:45 +07:00
0ae9560eea [vporn] Use urljoin for thumbnail 2016-12-16 23:57:51 +07:00
dc1f3a9f20 [vvvvid] do not cache the conn_id 2016-12-16 11:05:46 +01:00
7b1e80792b [vvvvid] Add new extractor(closes #5915) 2016-12-16 09:05:34 +01:00
38be3bc568 release 2016.12.15 2016-12-15 21:16:55 +07:00
d7ef47bffd [ChangeLog] Actualize 2016-12-15 21:15:45 +07:00
5c32a5be95 [openload] Recognize oload.tv URLs (#10408) 2016-12-15 17:51:26 +08:00
30918999f5 [facebook] Recognize .onion URLs (closes #11443) 2016-12-15 01:04:49 +08:00
069f918302 [vlive] Use live titles for live streams 2016-12-14 21:30:33 +07:00
89c63cc5f8 [vlive] Add video params extraction fallback and improve (closes #11375) 2016-12-14 21:05:50 +07:00
577748075b [vlive] Update extraction 2016-12-14 21:05:32 +07:00
67dcbc0add [canvas] extract dash formats 2016-12-13 17:59:22 +01:00
3a40f859b5 [melonvod] Improve (closes #11419) 2016-12-13 02:27:26 +07:00
e34c33614d [utils] Add convenience urljoin 2016-12-13 02:23:49 +07:00
abf3494ac7 [melonvod] Add extractor for vod.melon.com 2016-12-13 02:13:40 +07:00
3c1e9dc4ec release 2016.12.12 2016-12-12 01:44:50 +07:00
62faf9b55e [ChangeLog] Actualize 2016-12-12 01:41:08 +07:00
3530e0d3d9 [dplay] Use Safari user-agent for hls (closes #11418) 2016-12-12 00:58:08 +07:00
fb37eb25d9 [utils] Add common user agents map 2016-12-12 00:49:07 +07:00
d2d2495e16 [facebook] Detect login required error message 2016-12-11 01:40:30 +07:00
19b4900b7b [facebook] Improve video selection (closes #11390) 2016-12-11 01:22:01 +07:00
6ca478d44a [canalplus] Add another video id regex (closes #11399) 2016-12-11 00:45:27 +07:00
655cb545ab [mixcloud] Relax _VALID_URL (closes #11406) 2016-12-10 23:48:18 +07:00
f0b69fa91a [ctvnews] relax _VALID_URL regex(closes #11394) 2016-12-10 17:36:32 +01:00
8821a718cf [common] recognize hls manifests that contain video only formats(#11394) 2016-12-10 17:22:15 +01:00
0d7d9f9404 [rte] improve extraction(closes #10498)(closes #7746) 2016-12-10 16:34:01 +01:00
f41db40596 [prosiebensat1] extract dash formats 2016-12-10 13:29:51 +01:00
68601ef3ac [rts,srgssr] improve extraction for geo restricted videos(fixes #11089)(closes #4989) 2016-12-10 10:47:56 +01:00
18ece70c4d release 2016.12.09 2016-12-09 02:46:18 +07:00
9ed3495eae [ChangeLog] Actualize 2016-12-09 02:41:49 +07:00
6c20a0bb99 [openload] Fix extraction (closes #10408) 2016-12-09 02:15:16 +08:00
f43795e56b [pandoratv] PEP 8 and simplify 2016-12-07 23:50:10 +07:00
7441915b1e [pandoratv] Fix extraction (closes #11023) 2016-12-07 23:46:42 +07:00
283d1c6a8b [telebruxelles] extract all formats and add support for emission urls 2016-12-06 19:01:17 +01:00
875ddd7409 [bloomberg] Add another video id regex (closes #11371) 2016-12-06 00:41:03 +07:00
4afa4ff223 [1tv] Fix video id extraction 2016-12-05 23:28:57 +07:00
3ed81714d8 [fusion] Update ooyala id regex 2016-12-05 22:43:36 +07:00
4bd7d9d4ae [socks] Refine exception model for better error handling
1. ProxyError now inherits from socket.error instead of IOError

The only functions socks.py overrides are connect and connect_ex. In
Python 2.x and Python <= 3.2, socket functions raises socket.error. In
newer Python versions, those functions raises OSError instead. The name
socket.error is preserved as an alias of OSError for backward
compability. To keep socks.py compatible with Python's standard library,
it should raise the same exception as raw sockets.

See PEP 3151 (https://www.python.org/dev/peps/pep-3151/) for more
information about the change in Python 3.3.

2. Raise EOFError instead of IOError when the socket receives less data
than it expects

There's no common convention, but both ftplib and telnetlib raises
EOFError for similar situations. socks.py follows them.

Closes #11355

In #11355, only Python 2 is affected. In Python 3, both socket.error and
IOError are alias of OSError, so AbstractHTTPHandler.do_open correctly
catches the error and thus InfoExtractor._is_valid_url works fine.
2016-12-05 00:43:37 +08:00
9b5288c92a [1tv] Improve extraction and add support for playlists (closes #11335) 2016-12-04 23:35:21 +07:00
8344296619 [socks] Fix error reporting (#11355) 2016-12-03 21:53:41 +08:00
a94e7f4a0c [aenetworks] extract more formats(closes #11321) 2016-12-01 12:15:35 +01:00
d17bfe4095 [thisoldhouse] Recognize /tv-episode/ URLs and update _TESTS
Closes #11271
2016-12-01 14:56:52 +08:00
98b08f94b1 [README.md] Fix typo
Just a minor spelling mistake in the readme
2016-12-01 01:31:21 +07:00
73ec479c7d release 2016.12.01 2016-12-01 00:15:12 +07:00
f150530f4d [ChangeLog] Actualize 2016-12-01 00:13:06 +07:00
4c4765dba2 [soundcloud] Update client id (closes #11327) 2016-11-30 23:17:30 +07:00
f882554815 [comedcycentral] Give /shows/.+/full-episodes URLs to the COmedyCentralFullEpisodesIE 2016-11-30 11:52:19 +01:00
db75f14d8a [ruutu] Detect DRM videos 2016-11-30 04:19:38 +07:00
8b0d3ee64e [liveleak] Simplify and PEP 8 2016-11-29 23:42:19 +07:00
3779d524df [liveleak] Add support for youtube embeds 2016-11-29 23:37:30 +07:00
6303fc8204 [spike] Fix full episodes extraction 2016-11-29 23:06:01 +07:00
cc61fc3934 [comedycentral] Add new extractor for full-episodes
CC seems to have added yet another indirection for full episodes - the mgid is now only in a linked feed.
This may be a little brittle, but it's better than failing outright.
Plus, the current The Daily Show episode now works :)
2016-11-29 10:12:18 +01:00
c2530d3319 [teamfourstar] Simplify _VALID_URL and relax regexes 2016-11-28 23:22:29 +07:00
8953319916 [screenwavemedia] Remove extractor
Rewrite TeamFourStar and Normalboots extractors in terms of JWPlatform
2016-11-28 23:17:56 +07:00
51b1378eed Ignore and clean .swf files
Some videos on NicoNico are swf
2016-11-27 22:01:07 +08:00
2b380fc299 release 2016.11.27 2016-11-27 20:05:32 +07:00
294d4926d7 [ChangeLog] Actualize 2016-11-27 20:04:03 +07:00
83f1481baa [extractor/generic] Add support for webcaster.pro embeds 2016-11-27 19:56:32 +07:00
f25e1c8d8c [webcaster] Add support for webcaster.pro 2016-11-27 19:54:59 +07:00
6901673868 [azubu] Add support for azubu.uol.com.br (closes #11305) 2016-11-27 15:40:28 +07:00
560c8c6ec0 [viki] Prefer hls 2016-11-26 00:14:09 +07:00
9338a0eae3 [viki] Fix rtmp formats extraction (closes #11255) 2016-11-26 00:13:46 +07:00
74394b5e10 [puls4] Relax _VALID_URL (closes #11267) 2016-11-25 23:37:32 +07:00
1db058466d [vevo] Allow video info to fail in tests 2016-11-24 23:10:58 +07:00
e94eeb1dd3 [vevo] Simplify artists extraction 2016-11-24 23:09:35 +07:00
8b27d83e4e vevo: fixing naming when there are featured artists 2016-11-24 23:07:28 +07:00
8eb7b5c3f1 [mitele] Modernize and extract more metadata 2016-11-24 22:43:02 +07:00
b68599ed47 [mitele] Relax _VALID_URL 2016-11-24 21:57:53 +07:00
44444f0d3b [cbslocal] Support newyork.cbslocal.com
Closes #11285
2016-11-24 20:32:17 +08:00
c867adc68c [youtube:playlist] Pass disable_polymer in query (closes #11193, closes #11270) 2016-11-23 23:28:32 +07:00
3b5daf0736 release 2016.11.22 2016-11-22 22:32:16 +07:00
c8f56741dd [ChangeLog] Actualize 2016-11-22 22:29:37 +07:00
868630fbe5 [hellporno] Add support for hellporno.net and improve ext extraction 2016-11-22 22:16:10 +07:00
1d6ae5628f [amcnetworks] Recognize more BBC America URLs
Closes #11263
2016-11-22 20:40:57 +08:00
6334794f2a [funnyordie] Copy formats' metadata from hls and sort formats 2016-11-21 23:46:55 +07:00
4eece8ba57 [funnyordie] Improve extraction 2016-11-21 22:16:26 +07:00
2574721a81 Clean and ignore more file types
ape is another audio codec seen in kuwo. See
https://en.wikipedia.org/wiki/Monkey's_Audio
2016-11-21 12:50:13 +08:00
dbcc4a6b32 [CONTRIBUTING.md] Fix broken links (#11239) 2016-11-21 12:25:19 +08:00
0bb58a208b Merge pull request #11239 from josephfrazier/patch-1
[CONTRIBUTING.md] Fix broken link
2016-11-21 12:24:11 +08:00
dc6a9e4195 [README.md] Update link from generated CONTRIBUTING.md 2016-11-20 11:32:00 -05:00
8f8f182d0b [extractor/generic] Improve limelight embeds support 2016-11-20 02:13:21 +07:00
2176e466e0 Merge branch 'DarkstaIkers-master' 2016-11-20 00:07:35 +08:00
303b38fa84 [ChangeLog] Update for #9028 2016-11-20 00:06:44 +08:00
fb27d0ce5e Merge branch 'master' of https://github.com/DarkstaIkers/youtube-dl into DarkstaIkers-master 2016-11-20 00:05:11 +08:00
0aacd2deb1 [bandcamp] Fix free downloads extraction and extract all formats (closes #11067) 2016-11-19 04:18:21 +07:00
08ec95a6db [ChangeLog] Actualize 2016-11-19 03:10:20 +07:00
df46b19cb8 [toutv] Fix login form regex (closes #11223) 2016-11-19 01:56:31 +07:00
748a462fbe [twitter:card] Relax _VALID_URL (closes #11225) 2016-11-19 01:49:13 +07:00
c131fc3372 [tvanouvelles] Add extractor (closes #10616) 2016-11-18 01:16:33 +07:00
b25459b88a release 2016.11.18 2016-11-18 00:25:24 +07:00
5f75c4a4ad [ChangeLog] Actualize 2016-11-18 00:19:55 +07:00
689f31fde5 [devscripts/create-github-release] Fill release body from ChangeLog (closes #11094) 2016-11-18 00:17:46 +07:00
582be35847 Update coding style after pycodestyle 2.1.0
In pycodestyle 2.1.0, E305 was introduced, which requires two blank
lines after top level declarations, too.

See https://github.com/PyCQA/pycodestyle/issues/400

See also #10689; thanks @stepshal for first mentioning this issue and
initial patches
2016-11-17 19:45:42 +08:00
073d5bf583 [youtube:live] Relax _VALID_URL (closes #11164) 2016-11-16 23:15:19 +07:00
315cb86a95 Merge pull request #11210 from FooBarQuaxx/patch-2
Strip only args urls
2016-11-16 23:29:37 +08:00
b2fc1c4fb9 Add explanatory comment 2016-11-16 18:18:54 +03:00
d76767c90e [ChangeLog] Update after #11122 landed 2016-11-16 20:47:15 +08:00
eceba9f805 Merge pull request #11122 from kasper93/openload
[openload] Fix extraction.
2016-11-16 20:43:19 +08:00
MAA
d755396804 Strip only args urls 2016-11-16 09:00:30 +03:00
58355a3bf1 [vlive] Add test for #11203 2016-11-15 22:11:47 +07:00
49b69ad91c [vlive] Prefer locale over language for subtitles id 2016-11-15 22:07:17 +07:00
6b4dfa2819 release 2016.11.14.1 2016-11-14 02:48:15 +07:00
9f60134a9d [ChangeLog] Actualize 2016-11-14 02:46:12 +07:00
b3d4bd05f9 release 2016.11.14 2016-11-14 02:39:50 +07:00
dbffd00ba9 [ChangeLog] Actualize 2016-11-14 02:37:21 +07:00
a0758dfa1a [filmon] new extractor 2016-11-13 17:28:17 +01:00
50913b8241 [nrk] Improve geo restriction detection 2016-11-13 22:29:36 +07:00
7e08e2cab0 [nrk] Add X-Forwarded-For HTTP header in info dict 2016-11-13 22:28:29 +07:00
690355551c [downoader/fragment,f4m,hls] Add internal support for custom HTTP headers 2016-11-13 22:22:10 +07:00
754e6c8322 [nrk] Workaround geo restriction and improve error messages 2016-11-13 20:54:34 +07:00
e58609b22c [afreecatv] Add support for vod.afreecatv.com (closes #11174) 2016-11-13 06:02:26 +07:00
4ea4c0bb22 [extractor/common] Fix Bandwidth substitution in media template (closes #11175) 2016-11-13 05:43:34 +07:00
577281b0c6 [cda] Fix and improve extraction
Fixes #10929
2016-11-13 01:01:29 +07:00
3d2729514f [plays] Improve extraction and add support for embed URLs 2016-11-12 23:08:05 +07:00
f076d7972c [extractor/common] Improve thumbnail extraction from JSON-LD 2016-11-12 23:01:05 +07:00
cpm
8b1aeadc33 [plays] Fix extraction 2016-11-12 22:59:39 +07:00
95ad9ce573 [openload] Fix extraction.
aadecode code was restored from commit c1decda58c
with some optimizations (2x faster).

Fixes #10408
2016-11-11 15:36:57 +01:00
189935f159 [jsinterp] Fix function calls without arguments. 2016-11-11 15:36:57 +01:00
bc40b3a5ba [eagleplatform] Fix extraction (closes #11160) 2016-11-11 03:26:29 +07:00
3eaaa8abac [audioboom] Recognize /posts/ URLs (closes #11149) 2016-11-10 14:52:34 +08:00
db3367f43e release 2016.11.08.1 2016-11-08 22:30:53 +07:00
6590925c27 [ChangeLog] Actualize 2016-11-08 22:29:16 +07:00
4719af097c [extractors] Add forgotten import for espn:article 2016-11-08 22:27:02 +07:00
9946aa5ccf [franceculture] Fix extraction (closes #11140) 2016-11-08 22:26:33 +07:00
c58e07a7aa release 2016.11.08 2016-11-08 22:11:21 +07:00
f700afa24c [ChangeLog] Actualize 2016-11-08 22:09:03 +07:00
5d47b38cf5 [tmz:article] Fix extraction (closes #11052) 2016-11-08 21:53:41 +08:00
ebc7ab1e23 [espn] Fix extraction (closes #11041) 2016-11-08 00:29:12 +07:00
97726317ac [README.md] Mention HTTP headers and alternative way to obtain cookies and headers in -g FAQ 2016-11-07 23:53:22 +07:00
cb882540e8 [mitele] Fix extraction after website redesign (fixes #10824) 2016-11-07 11:13:59 +01:00
98708e6cbd [ard] Remove age restriction check (closes #11129) 2016-11-06 23:20:15 +07:00
b52c9ef165 [extractor/generic] Improve support for pornhub embeds (closes #11100) 2016-11-06 21:52:00 +07:00
e28ed498e6 [extractor/generic] Add support for redtube embds (closes #11099) 2016-11-06 21:42:41 +07:00
5021ca6c13 [redtube] Add support for embed URLs 2016-11-06 21:39:29 +07:00
37e7a71c6c [extractor/generic] Add support for drtuber embds (closes #11098) 2016-11-06 21:33:51 +07:00
f5c4b06f17 [drtuber] Fix title extraction 2016-11-06 21:29:15 +07:00
519d897049 [drtuber] Add support for embed URLs 2016-11-06 21:28:51 +07:00
b61cd51869 [yahoo] Add test and improve some content id regex 2016-11-06 21:16:33 +07:00
f420902a3b [yahoo] Add another content id regex (closes #11088) 2016-11-06 21:14:15 +07:00
de328af362 [toutv] Relax _VALID_URL (closes #11121) 2016-11-05 03:24:42 +07:00
b30e4c2754 release 2016.11.04 2016-11-04 22:07:54 +07:00
09ffe34b00 [ChangeLog] Actualize 2016-11-04 21:59:42 +07:00
640aff1d0c [anvato] Improve formats extraction 2016-11-04 21:45:24 +07:00
c897af8aac [cbslocal] Update test 2016-11-04 21:33:08 +07:00
f3c705f8ec [fox9] Add extractor (closes #11110) 2016-11-04 21:32:30 +07:00
f93ac1d175 [anvato] Extract more metadata 2016-11-04 21:17:56 +07:00
c4c9b8440c [extractor/common] Tolerate malformed RESOLUTION attribute in m3u8 manifests (closes #11113) 2016-11-04 05:02:31 +07:00
32f2627aed [vodlocker] Add another removed file pattern (closes #11106) 2016-11-03 22:22:40 +07:00
9d64e1dcdc [downloader/ism] Fix typo 2016-11-03 22:15:09 +07:00
10380e55de [downloader/ism] fix AVC Decoder Configuration Record creation in python 3 2016-11-03 16:08:57 +01:00
22979993e7 [vice] add coding cookie 2016-11-03 16:07:22 +01:00
b47ecd0b74 [vzaar] Add new extractor(closes #11093) 2016-11-03 12:50:41 +01:00
3a86b2c51e Ignore and clean .wav files 2016-11-03 18:55:55 +08:00
b811b4c93b [vice] add support for uplynk preplay videos(#11101) 2016-11-03 10:37:07 +01:00
f4dfa9a5ed [tubitv] fix extraction(closes #11061) 2016-11-03 09:04:20 +01:00
3b4b66b50c [shahid] add support for authentication(closes #11091) 2016-11-03 00:44:12 +01:00
4119a96ce5 [extractor/generic] Skip URLs we came from when delegating ISM extraction 2016-11-02 23:43:41 +07:00
26aae56690 [extractor/generic] Improve ISM extraction 2016-11-02 23:34:37 +07:00
4f9cd4d36f [radiocanada] extract subtitle(closes #11096) 2016-11-02 13:55:40 +01:00
cc99a77ac1 [extractor/generic] Add support for ISM manifests 2016-11-02 03:01:13 +07:00
8956d6608a release 2016.11.02 2016-11-02 02:39:36 +07:00
3365ea8929 [extractor/common] Remove unused code 2016-11-02 02:34:23 +07:00
a18aeee803 [ChangeLog] Actualize 2016-11-02 02:33:17 +07:00
1616f9b452 [extractor/common] Fix typo 2016-11-02 02:30:25 +07:00
02dc0a36b7 [utils] Introduce base_url 2016-11-02 02:30:18 +07:00
639e3b5c99 extract ISM formats in some of the extractors 2016-11-02 01:54:45 +07:00
b2758123c5 add Basic support for Smooth Streaming protocol(#8118) 2016-11-02 01:54:45 +07:00
f449c061d0 [nicknight] Improve extraction (closes #10769) 2016-11-02 01:35:53 +07:00
9c82bba05d [nickde] Improve extraction 2016-11-02 01:29:05 +07:00
e3577722b0 [nicknight] Add extractor 2016-11-02 01:25:59 +07:00
b82c33dd67 [extractor/common] Improve mpd base URL extraction (closes #10909, closes #11079) 2016-11-01 01:15:46 +07:00
e5a088dc4b [utils] Fix --match-filter for int-like strings (closes #11082) 2016-10-31 23:32:08 +07:00
2c6da7df4a release 2016.10.31 2016-10-31 01:36:53 +07:00
7e7a028aa4 [README.md] Fix a typo 2016-10-31 01:12:36 +07:00
e70a5e6566 release 2016.10.30 2016-10-30 18:24:49 +07:00
3bf55be466 [ChangeLog] Actualize 2016-10-30 18:19:29 +07:00
a901fc5fc2 [vessel] Add tests for #11068 2016-10-30 18:17:15 +07:00
cae6bc0118 [vessel] Improve video id extraction 2016-10-30 18:14:51 +07:00
d9ee2e5cf6 [facebook] Remove SWF params so that 1080P are detected
Closes #11073

In the provided link, SWF params give up to 720P, and VideoConfig
gives 1080P for both best and bestvideo. I guess all Facebook videos
supports HTML5 now, so I remove the old detection for SWF params
2016-10-30 18:20:55 +08:00
e1a0b3b81c [imgur] Recognize /r/ URLs (closes #11071) 2016-10-30 17:02:03 +08:00
2a048f9878 [beeg] Fix extraction (closes #11069) 2016-10-30 05:27:50 +07:00
ea331f40e6 Credit @Thor77 for jamendo (#10934) 2016-10-30 05:10:31 +07:00
f02700a1fa [openload] Fix extraction (#10408)
Thanks @TwelveCharzz again for studying openload codes
2016-10-29 18:00:30 +08:00
f3517569f6 [gvsearch] Modernize and fix page result request (closes #11051) 2016-10-28 23:19:59 +07:00
c725333d41 [ard] Fix typo 2016-10-27 03:59:00 +07:00
a5a8877f9c [adultswim] Fix extraction (closes #10979) 2016-10-27 02:16:48 +08:00
43c53a1700 [nobelprize] Add new extractor(closes #9999) 2016-10-26 18:15:23 +01:00
ec8705117a [hornbunny] Fix extraction (#10981) 2016-10-27 00:10:51 +08:00
3d8d44c7b1 [tvp] improve video id extraction(closes #10585) 2016-10-26 16:47:22 +01:00
88839f4380 release 2016.10.26 2016-10-26 19:55:09 +07:00
83e9374464 [ChangeLog] Actualize 2016-10-26 19:53:44 +07:00
773017c648 [rentv] Move rentv test from generic extractor and add only matching tests 2016-10-26 19:52:43 +07:00
777d90dc28 [rentv] Add new extractor(closes #10620) 2016-10-26 10:07:35 +01:00
3791d84acc [ard] Detect unavailable videos (closes #11018) 2016-10-25 21:21:47 +07:00
9305a0dc60 [vk] Fix extraction (closes #11022) 2016-10-25 21:05:29 +07:00
94e08950e3 release 2016.10.25 2016-10-25 03:19:36 +07:00
ee824a8d06 [ChangeLog] Actualize
[ci skip]
2016-10-25 02:51:07 +07:00
d3b6b3b95b [jamendo] Improve 2016-10-25 02:46:48 +07:00
b17422753f [jamendo] Add extractor 2016-10-25 02:43:03 +07:00
b0b28b8241 [ChangeLog] Actualize 2016-10-25 01:53:41 +07:00
81cb7a5978 Credit @azuwis for pandatv (#10736) 2016-10-25 01:51:46 +07:00
d2e96a8ed4 [pandatv] Extract m3u8, document reverse source and PEP 8 2016-10-25 01:51:37 +07:00
2e7c8cab55 [pandatv] Add new extractor 2016-10-25 01:46:02 +07:00
d7d4481c6a [movieclips] Fix _VALID_URL 2016-10-24 23:54:42 +07:00
5ace137bf4 [dotsub] Support vimeo embed (closes #10964) 2016-10-24 15:13:33 +08:00
9dde0e04e6 [litv] Fix extraction (#11006) 2016-10-23 23:23:40 +08:00
f16f8505b1 [vimeo] Delegate ondemand redirects to ondemand extractor (closes #10994) 2016-10-23 18:48:50 +07:00
9dc13a6780 [vivo] Fix extraction (closes #11003) 2016-10-23 18:07:56 +07:00
9aa929d337 [twitch:stream] Add support for rebroadcasts (closes #10995) 2016-10-23 17:20:45 +07:00
425f3fdfcb [pluralsight] Fix subtitles conversion (closes #10990) 2016-10-22 21:15:39 +07:00
e034cbc581 Merge branch 'johnhawkinson-stdin2' 2016-10-22 13:10:27 +08:00
5378f8ce0d [ChangeLog] Update for #10996 2016-10-22 13:08:56 +08:00
b64d04c119 [utils] Clarify for redirecting STDIN in get_exe_version() 2016-10-22 13:04:05 +08:00
00ca755231 [get_exe_version] Do version probes with <&-
When doing version probes for ffmpeg, do the
equivalent of calling it as:

    ffmpeg -version <&-

Where <&- is shell syntax for closing stdin before calling the
program. This is roughly equivalent to </dev/null without actually
opening /dev/null.

This prevents ffmpeg -version from hanging when run in the background.
Fixes #955.

The reason is that ffmpeg tries to manipulate stdin to set up terminal
characteristic, and that causes the kernel to suspend the parent
process (youtube-dl).

Note that closing stdin is achieved by calling subprocess.Popen() with
stdin set to subprocess.PIPE and without passing any input to
Popen.communicate(). This is somewhat subtle.
2016-10-22 00:34:08 -04:00
69c2d42bd7 release 2016.10.21.1 2016-10-21 04:57:28 +07:00
062e2769a3 [ChangeLog] Actualize 2016-10-21 04:53:26 +07:00
859447a28d [adobepass] PEP 8 2016-10-21 04:38:14 +07:00
f8ae2c7f30 [pluralsight] Process all clip URLs (closes #10984) 2016-10-21 04:35:32 +07:00
9ce0077485 release 2016.10.21 2016-10-21 03:08:42 +07:00
0ebb86bd18 [ChangeLog] Actualize 2016-10-21 03:07:03 +07:00
9df6b03caf [pluralsight] Adapt to new API (closes #10972) 2016-10-21 03:00:03 +07:00
8e2915d70b Revert "[postprocessor/embedthumbnail] Allow mkv to embed thumbnails"
This reverts commit 7360db05b4.

This commit was added as an attempt to fix #6046. Unfortunately, the fix
is completely wrong. As reported on #10359, embedded thumbnails are not
displayed in VLC, and Se7en on IRC reports that the embedded thumbnail
misleads mpv as well.

The correct way is using -attachment of ffmpeg, while the current
run_ffmpeg_multiple_files API can't handle it cleanly.
2016-10-20 15:07:19 +08:00
19e447150d [ChangeLog] Update for #10971 2016-10-20 04:19:33 +08:00
ad9fd84004 Merge pull request #10971 from kasper93/openload
[openload] Fix extraction.
2016-10-20 04:18:27 +08:00
60633ae9a0 [openload] Fix extraction.
Fixes #10408
2016-10-19 22:00:29 +02:00
a81dc82151 Credit @raleeper for the Comcast support (#10819) 2016-10-19 20:39:52 +01:00
9218a6b4f5 Merge pull request #10819 from raleeper/adobepass
[adobepass] Add Comcast
2016-10-19 20:16:24 +01:00
02af6ec707 [natgeo] extract m3u8 formats(closes #10959) 2016-10-19 19:39:41 +01:00
05b7996cab [ChangeLog] Fix typos and add unreleased version header 2016-10-20 00:00:53 +07:00
46f6052950 [adobepass] Add Comcast with fixed _download_webpage calls 2016-10-19 09:56:26 -07:00
c8802041dd release 2016.10.19 2016-10-19 23:55:16 +07:00
c7911009a0 [ChangeLog] Actualize 2016-10-19 23:53:37 +07:00
2b96b06bf0 [vidzi] Fix extraction (closes #10908, closes #10952) 2016-10-19 23:31:58 +07:00
06b3fe2926 [utils] Expose PACKED_CODES_RE 2016-10-19 23:28:49 +07:00
2c6743bf0f [README.md] Change 'guys' to 'people'
Folks at Ubuntu aren't all male so calling them 'guys' is a little odd.
2016-10-19 22:32:17 +07:00
efb6242916 [urplay] add supprt for urskola.se and fix subtitle extraction(closes #10915) 2016-10-19 15:05:39 +01:00
0384932e3d [extractor/common] try to extract non smil wowza mpd manifests 2016-10-19 14:57:12 +01:00
edd6074cea [extractor/common] detect f4m audio only formats 2016-10-19 14:42:48 +01:00
791d29dbf8 [orf] add subtitles support(closes #10939) 2016-10-19 11:34:46 +01:00
481cc7335c [youtube] Fix --no-playlist behavior for youtu.be/id URLs (closes #10896) 2016-10-19 03:27:18 +07:00
853a71b628 [nrk] Improve _VALID_URL 2016-10-19 03:02:14 +07:00
e2628fb6a0 [nrk] Relax _VALID_URL (closes #10928) 2016-10-19 02:59:44 +07:00
df4939b1cd [nytimes] Fix typo 2016-10-17 22:16:23 +07:00
0b94dbb115 [postprocessor/ffmpeg] PEP 8 2016-10-16 19:16:47 +07:00
8d76bdf12b [extractor/common] Mention podcast in series fields section 2016-10-16 18:37:17 +07:00
8204bacf1d Credit @johnhawkinson for nytimes podcasts (#10926) 2016-10-16 18:21:42 +07:00
47da782337 [nytimes] Improve (closes #10926) 2016-10-16 18:21:02 +07:00
74324a7ac2 [nytimes] Add support for podcasts 2016-10-16 17:31:55 +07:00
b0dfcab60a [pluralsight] Relax _VALID_URL (closes #10941) 2016-10-16 17:20:32 +07:00
bbd7706898 release 2016.10.16 2016-10-16 03:23:05 +07:00
112740e79f [ChangeLog] Actualize 2016-10-16 03:09:27 +07:00
c0b1e88895 [huajiao] Improve feed regex 2016-10-16 03:02:41 +07:00
7cdfbbf9b8 [extractors] Change import for theoperaplatform extractor 2016-10-16 02:55:38 +07:00
ac943d48d3 [Beatport] Update extractor name and tests 2016-10-16 02:33:43 +07:00
73498a8921 [ruutu] Add support for supla.fi 2016-10-16 02:31:56 +07:00
9187ee4f19 [README.md] Improve grammar 2016-10-16 02:26:06 +07:00
2273e2c530 [postprocessor/ffmpeg] Return correct filepath and ext in updated information in FFmpegExtractAudioPP
Return correct audio's filepath and ext instead of the video's when extracting audio and audio file already exists.
2016-10-16 02:12:03 +07:00
4b492e3579 [theoperaplatform] Rename, fix _VALID_URL and fix test 2016-10-16 00:24:06 +07:00
9c4258bcec [theoperaplatform] Add extractor 2016-10-16 00:21:15 +07:00
ea8aefd1d7 [lynda] Fix height for prioritized streams 2016-10-16 00:08:46 +07:00
6edfc40a0e [lynda] Add fallback extraction scenario 2016-10-16 00:07:40 +07:00
68d9561ca1 [lynda] Switch to https (closes #10916) 2016-10-15 23:56:09 +07:00
cfc0e7c82b Credit @pyx for the Huajiao extractor (#10917) 2016-10-15 16:48:35 +08:00
4102e64051 Merge branch 'pyx-huajiao' 2016-10-15 14:56:00 +08:00
f605242bfc [ChangeLog] Update for #10917 2016-10-15 14:55:36 +08:00
d32fa0f12c [huajiao] Coding style 2016-10-15 14:53:53 +08:00
a347a0d088 Merge branch 'huajiao' of https://github.com/pyx/youtube-dl into pyx-huajiao 2016-10-15 14:53:05 +08:00
77c5b98dcd [crunchyroll] Skip an invalid _TEST 2016-10-15 14:36:07 +08:00
88ebefc054 [cmt] Fix mgid extraction (closes #10813)
The example in #10813 requires TV provider authentication in Firefox,
while youtube-dl can download it directly with an US proxy.

I'm not sure whether the mgid fix is cmt-specific or it applies to all
mtv-based sites. I keep it in cmt.py until similar patterns are found in
other websites.
2016-10-15 14:27:15 +08:00
2e638d7bca Made optional fields optional 2016-10-14 14:12:06 -04:00
a26b174c61 [safari:course] Add support for techbus.safaribooksonline.com 2016-10-15 00:29:33 +07:00
73c801d660 [orf:tvthek] Fix extraction and modernize (closes #10898) 2016-10-14 23:43:09 +07:00
dff5107b68 README.md: fix alrady typo 2016-10-14 23:21:08 +07:00
8c3e448e80 [clipfish] Update _TEST; the old one is gone 2016-10-15 00:12:21 +08:00
2ecbd2ad6f [chirbit:profile] Fix extraction 2016-10-15 00:01:46 +08:00
62a0b86e4f [carambatv] Fix extraction
The video requested in #9815 now has videomore embeds.
2016-10-14 23:43:18 +08:00
146969e05b [videomore] Support <iframe> embed videos
Seen in CarambaTVPage
2016-10-14 23:42:11 +08:00
e2004ccaf7 [canalplus] Fix video_id and update _TESTS
Some tests are gone, and some redirect to different videos
2016-10-14 20:26:12 +08:00
a5f8473145 [cbsinteractive] Fix extraction for cnet.com 2016-10-14 18:20:01 +08:00
b7f59a3bf6 [huajiao] Add new extractor 2016-10-13 21:51:26 -04:00
580d411931 [parliamentliveuk] Recognize lower case URLs
Closes #10912

Seems parliamentliveuk matches URLs case-insentive. For example this URL
also works:
http://parliamentlive.tv/EvEnt/Index/3F24936f-130f-40bf-9a5d-b3d6479da6a4
2016-10-14 00:44:28 +08:00
5c4bfd4da5 release 2016.10.12 2016-10-12 21:30:05 +07:00
7104ae799c [ChangeLog] Actualize 2016-10-12 21:25:04 +07:00
bcd6276520 [downloader/common] Remove debug output 2016-10-12 21:22:33 +07:00
591e384552 [streamable] Remove debug output 2016-10-12 21:22:12 +07:00
9feb1c9731 [dailymotion] Fix extraction and update _TESTS
Closes #10901

Seems all videos use player V5 syntax now
2016-10-12 21:45:49 +08:00
a093cfc78b [vimeo:review] Fix extraction (#10900)
Now Vimeo Review videos uses React. Thanks @davekaro for analyzing the
problem!
2016-10-12 01:48:06 +08:00
6f20b65e72 [test/test_http] Update tests
After switching to HTML5 extraction helpers in generic.py, the result
info_dict is always a playlist.
2016-10-12 01:41:41 +08:00
cea364f70c [extractor/common] Support HTML media elements without child nodes 2016-10-12 01:40:28 +08:00
55642487f0 [nhl] Skip invalid m3u8 formats (closes #10713) 2016-10-11 20:50:52 +08:00
3d643f4cec [hbo] Add HBOEpisodeIE (#10892) 2016-10-11 17:46:52 +08:00
c452e69d3d [footyroom] Fix extraction and update _TESTS (closes #10810) 2016-10-11 17:46:13 +08:00
555787d717 [streamable] Add helper for extracting embedded videos 2016-10-11 17:44:35 +08:00
f165ca70eb [abc.net.au:iview] Fix for non-series videos (closes #10895) 2016-10-11 12:53:27 +08:00
27b8d2ee95 [hbo] Add display_id and another test (#10892) 2016-10-11 12:41:44 +08:00
71cdcb2331 [hbo] Support episode pages (closes #10892) 2016-10-11 12:30:35 +08:00
176006a120 [allocine] Fix for /video/ videos (closes #10860) 2016-10-09 19:42:42 +08:00
65f4c1de3d [allocine] Fix extraction (closes #10860)
I change the URL of the third test case, because now the original URL
does not contain a video anymore, and there's no easy to get the real
URL from the /film/ one.
2016-10-09 18:58:15 +08:00
b0082629a9 [nextmedia] Support action news (動新聞) on Apple Daily 2016-10-09 18:42:15 +08:00
8204c73352 [Makefile] Fix for GNU make < 4 (closes #9387)
Shell assignment operator in BSD make != is ported to GNU make in
version 4.0, so 3.x doesn't work. I choose to drop BSD make support as
installing GNU make on *BSD systems is easier than installing newer GNU
make.
2016-10-09 18:24:45 +08:00
2b51dac1f9 [slutload] Fix test and simplify 2016-10-09 01:17:38 +07:00
f68901e50a [reverbnation] Eliminate code duplication in thumbnails extraction 2016-10-09 01:02:35 +07:00
3adb9d119e [reverbnation] Modernize 2016-10-09 01:00:38 +07:00
1dd58e14d8 [lego] improve info extraction and bypass geo restriction(closes #10872) 2016-10-08 08:33:18 +01:00
dd4291f729 release 2016.10.07 2016-10-07 22:25:30 +07:00
888f8d6ba4 [ChangeLog] Actualize 2016-10-07 22:23:16 +07:00
f475e88121 [vimeo] PEP 8
[ci skip]
2016-10-07 22:15:26 +07:00
3c6b3bf221 [iprima] detect geo restriction 2016-10-07 15:53:16 +01:00
38588ab977 [facebook] Fix for new handleServerJS syntax (closes #10846)
According to the dump file in #10846, handleServerJS() now accepts
an optional second argument. It's a string from available dump files.
2016-10-07 20:04:49 +08:00
85bcdd081c [extractors] Add MmsIE 2016-10-07 19:31:26 +08:00
9dcd6fd3aa [generic,commonprotocols] Move mms suuport from GenericIE
And use _generic_* helpers in those extractors
2016-10-07 19:24:22 +08:00
98763ee354 [extractor/common] Add id and title helpers for generic IEs 2016-10-07 19:20:53 +08:00
3d83a1ae92 [generic] Support direct MMS links (closes #10838) 2016-10-07 17:50:45 +08:00
c0a7b9b348 Revert "[Makefilea] Fix for GNU make < 4"
This reverts commit 831a34caa2.

The reverted commit breaks lazy extractors.
2016-10-07 16:03:34 +08:00
831a34caa2 [Makefilea] Fix for GNU make < 4
Closes #9387

The shell assignment operator != was introduced in GNU make 4.0, or
specifically the commit in [1]. This fix removes such usages and
fallback to a more portable syntax. Tested with:

* GNU make 3.82 on CentOS 7.2
* bmake 20150910 on CentOS 7.2, source RPM from Fedora 24 [2]
* GNU make 4.2.1 on Arch Linux (Arch official package)
* bmake 20160926 on Arch Linux (Arch official package)
* GNU make 3.82 on Arch Linux (Compiled from source)
* Apple bsdmake-24 on macOS Sierra, binary package from Homebrew

Thanks @bdeyal for the feedback of the first tests

[1] http://git.savannah.gnu.org/cgit/make.git/commit/?id=b34438bee83ee906a23b881f257e684a0993b9b1
[2] http://koji.fedoraproject.org/koji/buildinfo?buildID=716769
2016-10-07 03:28:41 +08:00
09b9c45e24 [generic] Add support for multiple vimeo embeds (Closes #10862) 2016-10-06 23:22:52 +07:00
33898fb19c [nzz] Add new extractor(#4407) 2016-10-06 10:45:57 +01:00
017eb82934 [npo] detect geo restriction 2016-10-05 18:27:02 +01:00
b1d798887e [npo] Add support for 2doc.nl (Closes #10842) 2016-10-05 23:43:08 +07:00
0a33bb2cb2 Rename "Steffan 'Ruirize' James" to "Steffan Donal"
Legal name change!
2016-10-05 03:32:14 +07:00
185744f92f [lego] Add new extractor(closes #10369) 2016-10-04 10:30:57 +01:00
7232e54813 [tonline] Add new extractor(#10376) 2016-10-04 08:00:25 +01:00
6eb5503b12 [techtalks] Relax _VALID_URL 2016-10-04 02:54:36 +07:00
539c881bfc [techtalks] Allow URL-s with name part omitted. 2016-10-04 02:52:33 +07:00
c1b2a0858c [youtube:live] Extend _VALID_URL (Closes #10839) 2016-10-04 02:10:23 +07:00
215ff6e0f3 [theweatherchannel] Add new extractor(closes #7188) 2016-10-03 18:20:34 +01:00
dcdb292fdd Unify coding cookie 2016-10-03 23:44:29 +07:00
c1084ddb0c [thisoldhouse] Add new extractor(closes #10837) 2016-10-03 15:27:09 +01:00
ee5de4e38e [nhl] Add support for wch2016.com (Closes #10833) 2016-10-03 00:54:02 +07:00
25291b979a Merge pull request #10829 from TRox1972/pornoxo_improve
[pornoxo] Use JWPlatform to improve metadata extraction
2016-10-02 20:19:34 +08:00
567a5996ca [pornoxo] Use JWPlatform to improve metadata extraction 2016-10-02 13:07:02 +02:00
6c152ce20f release 2016.10.02 2016-10-02 15:58:00 +07:00
26406d33c7 [ChangeLog] Actualize 2016-10-02 15:56:33 +07:00
703b3afa93 [amcnetworks] Skip a restricted _TEST 2016-10-02 14:25:06 +08:00
99ed78c79e [jwplatform] Support DASH streams 2016-10-02 14:07:49 +08:00
fd15264172 [jwplatform] Support old-style jwplayer playlists 2016-10-02 13:47:06 +08:00
bd26441205 [utils] Fix xattr error handling 2016-10-02 03:03:41 +08:00
b19e275d99 [__init__] Fix lost xattr if --embed-thumbnail used
Reported at
https://github.com/rg3/youtube-dl/issues/9054#issuecomment-250451823
2016-10-02 02:12:14 +08:00
f6ba581f89 [byutv:event] Add extractor 2016-10-02 00:50:07 +07:00
6d2549fb4f [byutv] Fix id and display id 2016-10-02 00:44:54 +07:00
4da4516973 [byutv] Rely on _match_id and _parse_json 2016-10-02 00:41:18 +07:00
e1e97c2446 [periscope:user] Fix extraction (Closes #10820) 2016-10-01 22:50:47 +07:00
53a7e3d287 [utils] Support xattr as well as pyxattr
Closes #9054

There are two xattr packages in Python, pyxattr [1] and xattr [2]. They
have different APIs.

In old days pyxattr supports Linux only and xattr supports Linux, Mac,
FreeBSD and Solaris, and pyxattr supports Linux only. Recently pyxattr
adds support for Mac OS X. [3]

An old version of [2] is shipped with Mac OS X. However, some Linux
distributions have pyxattr only, for example PLD-Linux [4] and old Arch
Linux. [5] As a result, supporting both is the way to go.

[1] https://github.com/iustin/pyxattr
[2] https://github.com/xattr/xattr
[3] https://github.com/iustin/pyxattr/pull/9
[4] https://github.com/rg3/youtube-dl/issues/5498
[5] https://git.archlinux.org/svntogit/community.git/commit/?id=427c4c76401e386d865ccddea4fbfdc74df80492
    https://git.archlinux.org/svntogit/community.git/commit/?id=59b40da7b69622a6761d364a8b07909e9cccaa56
    python-xattr is added on 2016/06/29 while pyxattr is there for more
    than 6 years
2016-10-01 20:13:04 +08:00
d54739a2e6 [downloader/http] xattr values should be bytes 2016-10-01 19:58:13 +08:00
63e0fd5bcc Merge pull request #10818 from TRox1972/criterion_match_id
[criterion] Rely on _match_id, improve regex and add thumbnail to test
2016-10-01 19:49:18 +08:00
9c51a24642 [criterion] Rely on _match_id, improve regex and add thumbnail to test 2016-10-01 13:46:48 +02:00
9bd7bd0b80 [twitch] Skip a 404 test 2016-10-01 16:38:47 +08:00
4a76b73c6c Merge pull request #10817 from TRox1972/clubic_match_id
[clubic] Rely on _match_id and _parse_json
2016-10-01 16:20:12 +08:00
e295618f9e [dctp] Fix extraction (closes #10734) 2016-10-01 15:22:48 +08:00
d7753d1948 [downloader/http] Use write_xattr function for --xattr-set-filesize 2016-10-01 14:47:20 +08:00
eaf9b22f94 [clubic] Rely on _match_id and _parse_json 2016-09-30 20:03:25 +02:00
a1001f47fc [instagram] PEP 8 2016-10-01 00:16:08 +07:00
1609782258 [Instagram] Extract video dimensions 2016-10-01 00:13:34 +07:00
de6babf922 [tvland] Extend _VALID_URL (Closes #10812) 2016-09-30 22:30:34 +07:00
b0582fc806 [vgtv] Add support for tv.aftonbladet.se (Closes #10800) 2016-09-30 00:15:09 +07:00
af33dd8ee7 [aftonbladet] Remove extractor 2016-09-30 00:13:03 +07:00
70d7b323b6 [vk] Improve view count extraction 2016-09-29 23:52:29 +07:00
a7ee8a00f4 [vk] Extract timestamp (Closes #10760) 2016-09-29 23:52:29 +07:00
c6eed6b8c0 [utils] Lower priority for rare date formats and add tests 2016-09-29 23:52:29 +07:00
3aa3953d28 [vk] Fix date and view count extraction. 2016-09-29 23:52:29 +07:00
efa97bdcf1 Move write_xattr to utils.py
There are some other places that use xattr functions. It's better to
move it to a common place so that others can use it.
2016-09-30 00:28:32 +08:00
475f8a4580 [vk] Add support for running live streams (Closes #10799) 2016-09-29 23:21:39 +07:00
93aa0b6318 [vk] Add support for finished live streams (#10799) 2016-09-29 23:04:10 +07:00
0ce26ef228 Merge pull request #10788 from TRox1972/instagram_comments
[Instagram] Extract comments
2016-09-29 21:54:39 +08:00
0d72ff9c51 [leeco] Recognize more Le Sports URLs (#10794) 2016-09-29 21:39:35 +08:00
a56e74e271 [Instagram] Extract comments 2016-09-28 19:32:40 +02:00
f533490bb7 [ketnet] Extract mzsource formats (#10770) 2016-09-28 22:58:25 +07:00
8bfda726c2 [limelight:media] improve http formats extraction 2016-09-28 16:34:27 +01:00
8f0cf20ab9 release 2016.09.27 2016-09-27 23:09:46 +07:00
c8f45f763c [ChangeLog] Remove duplicate 2016-09-27 23:03:00 +07:00
dd2cffeeec [ChangeLog] Actualize 2016-09-27 22:43:35 +07:00
cdfcc4ce95 [mtv] Improve _VALID_URL 2016-09-27 22:27:10 +07:00
e384552590 [vk] Add support for dailymotion embeds
Fixes #10661
2016-09-27 21:58:14 +07:00
1a2fbe322e [periscope] Treat timed_out state as finished stream 2016-09-27 21:55:51 +07:00
f9dd86a112 [npo] Clarify IE_NAMEs (Closes #10775) 2016-09-27 21:37:33 +07:00
2342733f85 fix tests related to 1978540a5122c53012e17a78841f3da0df77fd34(closes #10774) 2016-09-27 15:31:25 +01:00
93933c9819 [awaan:video] fix test(closes #10773) 2016-09-27 15:31:25 +01:00
d75d9e343e [einthusan] Fix extraction (closes #10714) 2016-09-27 14:38:41 +08:00
72c3d02d29 [promptfile] Improve and modernize 2016-09-26 23:39:54 +07:00
d3dbb46330 [promptfile] Fix extraction (Closes #10634) 2016-09-26 23:20:58 +07:00
fffb9cff94 [kaltura] Speed up embed regexes (#10764) 2016-09-26 22:15:58 +07:00
d3c97bad61 Ignore and cleanup 3gp files 2016-09-26 14:14:37 +08:00
2d5b4af007 [extractors] Add import for anderetijden extractor 2016-09-25 23:30:57 +07:00
f1ee462c82 [PULL_REQUEST_TEMPLATE.md] Fix typo 2016-09-25 22:38:36 +07:00
5742c18bc1 [npo] Add support for anderetijden.nl (Closes #10754) 2016-09-25 22:26:14 +07:00
ddb19772d5 [vpro] Fix playlist title extraction and update tests 2016-09-25 22:26:06 +07:00
a3d8b38168 [npo] Generalize playlist extractors 2016-09-25 22:26:00 +07:00
e590b7ff9e [PULL_REQUEST_TEMPLATE.md] Add checkable Improvement options PR's purpose 2016-09-25 18:09:46 +07:00
f3625cc4ca [PULL_REQUEST_TEMPLATE.md] Add Unlicense notice 2016-09-25 18:08:35 +07:00
2d3d29976b [youtube] Change test URLs from http to https 2016-09-25 17:45:24 +07:00
493353c7fd [prosiebensat1] Add support for advopedia 2016-09-25 06:25:57 +07:00
0a078550b9 [prosiebensat1] Improve _VALID_URL 2016-09-25 06:19:17 +07:00
f92bb612c6 [mwave] Relax _VALID_URLs (Closes #10735, closes #10748) 2016-09-25 06:14:32 +07:00
ddde91952f [prosiebensat1] Fix playlist support (Closes #10745) 2016-09-25 05:36:18 +07:00
63c583eb2c [prosiebensat1] Add support for sat1gold (#10745) 2016-09-25 04:43:10 +07:00
7fd57de6fb [cbsnews:livevideo] fix extraction and extract m3u8 formats 2016-09-24 22:01:33 +01:00
e71a450956 [common] add hdcore sign to akamai f4m formats 2016-09-24 21:55:53 +01:00
27e99078d3 [brightcove:new] add support for live streams 2016-09-24 15:39:48 +01:00
6f126d903f [download/hls] Delegate downloading to ffmpeg for live streams 2016-09-24 15:39:47 +01:00
7518a61d41 [soundcloud] Fix typo in playlist base class name 2016-09-24 19:29:49 +07:00
8e45e1cc4d [soundcloud] Generalize playlist entries extraction (#10733) 2016-09-24 19:18:01 +07:00
f0bc5a8609 [twitter] Support Periscope embeds (closes #10737)
Also update _TESTS
2016-09-24 20:00:29 +08:00
a54ffb8aa7 [mtv] add common IE_NAME prefix for MTVIE and MTVVideoIE 2016-09-24 10:50:14 +01:00
8add4bfecb [mtv] add support for new website urls(closes #8169)(closes #9808) 2016-09-24 10:42:20 +01:00
0711995bca [openload] Support subtitles (closes #10625) 2016-09-24 14:27:08 +08:00
5968d7d2fe [extractor/common] Improved support for HTML5 subtitles
Ref: #10625

In a strict sense, <track>s with kind=captions are not subtitles. [1]
openload misuses this attribute, and I guess there will be more
examples, so I add it to common.py.

Also allow extracting information for subtitles-only <video> or <audio>
tags, which is the case of openload.

[1] https://www.w3.org/TR/html5/embedded-content-0.html#attr-track-kind
2016-09-24 14:20:42 +08:00
e6332059ac release 2016.09.24 2016-09-24 02:16:47 +07:00
8eec691e8a [ChangeLog] Actualize 2016-09-24 02:12:49 +07:00
24628cf7db [soundcloud:playlist] Provide video id for playlist entries (Closes #10733) 2016-09-24 02:01:01 +07:00
71ad00c09f [prosiebensat1] Add support for kabeleinsdoku (Closes #10732) 2016-09-23 21:08:16 +07:00
45cae3b021 [cbs] extract info from thunder videoPlayerService(closes #10728) 2016-09-22 19:28:22 +01:00
4ddcb5999d [openload] Fix extraction (closes #10408, closes #10727)
Thanks to @daniel100097 for providing a working version
2016-09-23 01:47:51 +08:00
628406db96 [Makefile] Cleanup files from fragment-based downloaders 2016-09-23 01:13:56 +08:00
e3d6bdc8fc [ustream] Support HLS streams (closes #10698) 2016-09-23 01:11:13 +08:00
0a439c5c4c [udemy] Stringify video id 2016-09-22 21:48:53 +07:00
1978540a51 [ooyala] extract all hls formats 2016-09-21 21:49:52 +01:00
12f211d0cb [videomore] Fix embed regex 2016-09-21 22:51:36 +07:00
3a5a18705f [adobepass] add support MSO that depend on watchTVeverywhere(closes #10709) 2016-09-21 15:57:27 +01:00
1ae0ae5db0 [cartoonnetwork] add support Adobe Pass auth 2016-09-20 18:52:00 +01:00
f62a77b99a [soundcloud] Modernize 2016-09-20 21:56:57 +07:00
4bfd294e2f [soundcloud] Extract license metadata 2016-09-20 21:56:57 +07:00
e33a7253b2 [fox] add support for Adobe Pass auth(closes #8584) 2016-09-20 15:52:23 +01:00
c38f06818d add support for Adobe Pass auth in tbs,tnt and trutv extractors(fixes #10642)(closes #10222)(closes #10519) 2016-09-20 11:55:30 +01:00
cb57386873 release 2016.09.19 2016-09-19 02:58:32 +07:00
59fd8f931d [ChangeLog] Actualize 2016-09-19 02:57:14 +07:00
70b4cf9b1b [crunchyroll] Check if already logged in (Closes #10700) 2016-09-19 02:50:06 +07:00
cc764a6da8 [twitch:stream] Remove fallback to profile extraction when stream is offline
Main page does not contain profile videos anymore
2016-09-18 19:10:18 +07:00
d8dbf8707d [thisav] Improve title extraction (closes #10682)
I didn't add a test case as the one in #10682 looks like a copyrighted
product.
2016-09-18 18:35:38 +08:00
a1da888d0c [vyborymos] Improve station info extraction 2016-09-18 17:30:55 +07:00
3acff9423d release 2016.09.18 2016-09-18 17:16:55 +07:00
9ca93b99d1 [ChangeLog] Actualize 2016-09-18 17:15:22 +07:00
14ae11efab [vyborymos] Add extractor (Closes #10692) 2016-09-18 16:56:40 +07:00
190d2027d0 [xfileshare] Add title regex for streamin.to and fallback to video id (Closes #10646) 2016-09-18 07:22:06 +07:00
26394d021d [globo:article] Add support for multiple videos (Closes #10653) 2016-09-17 23:34:10 +07:00
30d0b549be [extractor/common] Add manifest_url for hls and hds formats 2016-09-17 21:33:38 +07:00
86f4d14f81 Refactor fragments interface and dash segments downloader
- Eliminate segment_urls and initialization_url
+ Introduce manifest_url (manifest may contain unfragmented data in this case url will be used for direct media URL and manifest_url for manifest itself correspondingly)
* Rewrite dashsegments downloader to use fragments data
* Improve generic mpd extraction
2016-09-17 20:35:22 +07:00
21d21b0c72 [svt] Fix DASH formats extraction 2016-09-17 19:25:31 +07:00
b4c1d6e800 [extractor/common] Expose fragments interface for dashsegments formats 2016-09-17 18:31:18 +07:00
a0d5077c8d [extractor/common] Introduce fragments interface 2016-09-17 18:31:09 +07:00
584d6f3457 [thisav] Recognize jwplayers (closes #10447) 2016-09-17 18:46:43 +08:00
e14c82bd6b [jwplatform] Use js_to_json to detect more JWPlayers 2016-09-17 18:45:08 +08:00
c51a7f0b2f [franceinter] Fix upload date extraction 2016-09-17 15:44:37 +07:00
d05ef09d9d [mangomolo] fix domain regex 2016-09-17 08:11:01 +01:00
30d9e20938 [postprocessor/ffmpeg] apply FFmpegFixupM3u8PP only for videos with aac codec(#5591) 2016-09-16 22:06:55 +01:00
fc86d4eed0 [mangomolo] fix typo 2016-09-16 20:10:47 +01:00
7d273a387a [mangomolo] add support for Mangomolo embeds 2016-09-16 19:31:39 +01:00
6ad0219556 [common] add helper method for Wowza Streaming Engine format extraction 2016-09-16 19:30:38 +01:00
98b7506e96 [toutv] add support for authentication(closes #10669) 2016-09-16 17:40:15 +01:00
52dc8a9b3f [franceinter] Fix upload date extraction 2016-09-16 22:02:59 +07:00
9d8985a165 [tv4] Fix hls and hds formats (Closes #10659) 2016-09-16 00:54:34 +07:00
f5e008d134 release 2016.09.15 2016-09-15 23:46:11 +07:00
e6bf3621e7 [ChangeLog] Actualize 2016-09-15 23:31:16 +07:00
490b755769 Improve some id regexes 2016-09-15 23:12:58 +07:00
1dec2c8a0e [adobepass] Change mvpd cache section name
In order to better emphasize it's relation to Adobe Pass
2016-09-15 22:47:45 +07:00
dcce092e0a [extractor/common] Simplify _get_netrc_login_info and carry long lines 2016-09-15 22:35:12 +07:00
32443dd346 [extractor/common] Update _get_login_info's comment 2016-09-15 22:34:29 +07:00
2133565cec [extractor/common] Simplify _get_login_info 2016-09-15 22:26:37 +07:00
1da50aa34e [YoutubeDL] Improve Adobe Pass options' wording 2016-09-15 22:24:55 +07:00
d2522b86ac [options] Actually print Adobe Pass options sections in --help 2016-09-15 22:18:31 +07:00
537f753399 [options] Improve Adobe Pass wording 2016-09-15 22:17:17 +07:00
c849836854 [utils] Improve _hidden_inputs 2016-09-15 21:54:48 +07:00
eb5b1fc021 [crunchyroll] Fix authentication (Closes #10655) 2016-09-15 21:53:35 +07:00
95be29e1c6 [twitch] Fix api calls (Closes #10654, closes #10660) 2016-09-15 20:58:02 +07:00
c035dba19e [bellmedia] add support for more sites 2016-09-15 08:12:12 +01:00
87148bb711 [adobepass] rename --ap-mso-list option to --ap-list-mso 2016-09-14 20:21:09 +01:00
797c636bcb [ap] improve adobe pass names and parse error handling 2016-09-14 18:58:47 +01:00
0002962f3f [franceinter] Improve extraction (Closes #10538) 2016-09-14 23:59:38 +07:00
3e4185c396 [utils] Use native french month names 2016-09-14 23:59:38 +07:00
f6717dec8a [utils] Improve month_by_name and add tests 2016-09-14 23:59:38 +07:00
a942d6cb48 [utils,franceinter] Add french months' names and fix extraction
Update of the "FranceInter" radio extractor : webpages HTML structure
had changed, the extractor didn't work. So I updated this extractor to
get the mp3 URL and all details.
2016-09-14 23:59:38 +07:00
961516bfd1 [kwuo:song] Improve error detection (closes #10650) 2016-09-15 00:56:15 +08:00
6db354a9f4 [kuwo] Update _TESTS 2016-09-15 00:53:04 +08:00
353f340e11 [go] fix typo 2016-09-14 17:22:42 +01:00
014b7e6b25 [go] add support for free full episodes(#10439) 2016-09-14 17:08:25 +01:00
925194022c Improve some _VALID_URLs 2016-09-14 22:47:21 +07:00
b690ea15eb [viafree] Fix test 2016-09-14 22:45:23 +07:00
5712c0f426 [adobepass] remove unnecessary option 2016-09-14 16:37:21 +01:00
86d68f906e [bilibili] Fix extraction for videos without backup_url (#10647) 2016-09-14 22:11:49 +08:00
4875ff6847 [bilibili] Remove copyrighted test cases
I can't find any English or Chinese material that claims BiliBili has
bought legal redistribution permissions for copyrighted products from
copyrighted holders.

References for removed test cases:
"刀语": https://en.wikipedia.org/wiki/Katanagatari, by White Fox
"哆啦A梦": https://en.wikipedia.org/wiki/Doraemon, by Shin-Ei Animation
"岳父岳母真难当": https://en.wikipedia.org/wiki/Serial_(Bad)_Weddings, by Les films du 24
"混沌武士": https://en.wikipedia.org/wiki/Samurai_Champloo, by Manglobe

I shouldn't have added them to _TESTS
2016-09-14 22:09:43 +08:00
1b6712ab23 [adobepass] add specific options for adobe pass authentication
- add --ap-username and --ap-password option to specify
TV provider username and password in the cmd line
- add --ap-retries option to limit the number of retries
- add --list-ap-msi-ids to list the supported TV Providers
2016-09-13 22:16:01 +01:00
8414c2da31 [adobepass] PEP 8 2016-09-13 23:22:16 +07:00
45396dd2ed [nhk] Fix extraction (Closes #10633) 2016-09-13 23:20:25 +07:00
7a7309219c [adobepass] add an option to specify mso_id and support for ROGERS TV Provider(closes #10606) 2016-09-12 23:39:35 +01:00
fcba157e80 [ISSUE_TEMPLATE_tmpl.md] Fix typo 2016-09-12 23:29:43 +07:00
a6ccc3e518 [safari] Improve ids regexes (#10617) 2016-09-12 23:05:52 +07:00
1d16035bb4 [kaltura] Improve audio detection 2016-09-12 22:43:45 +07:00
e8bcd982cc [kaltura] Skip chun format 2016-09-12 22:33:00 +07:00
a5ff05df1a [extractor/generic] Add vimeo embed that requires Referer passed 2016-09-12 21:49:31 +07:00
d002e91986 [vimeo:ondemand] Pass Referer along with embed URL (#10624) 2016-09-12 21:48:45 +07:00
546edb2efa [ISSUE_TEMPLATE_tmpl.md] Fix typo 2016-09-12 21:01:31 +07:00
be45730226 [nbc] Add new extractor for NBC Olympics (#10295, #10361) 2016-09-12 02:55:15 +08:00
ee7e672eb0 [tube8] Remove proxy settings from test 2016-09-11 23:46:50 +07:00
0307d6fba6 release 2016.09.11.1 2016-09-11 23:33:20 +07:00
fc150cba1d [devscripts/release.sh] Add missing fi 2016-09-11 23:32:01 +07:00
d667ab7fad [ChangeLog] Actualize 2016-09-11 23:30:18 +07:00
eb87d4545a [devscripts/release.sh] Add ChangeLog reminder prompt 2016-09-11 23:29:25 +07:00
1c81476cbb release 2016.09.11 2016-09-11 23:20:09 +07:00
bc9186c882 [tvplay] Remove unused import 2016-09-11 22:51:12 +07:00
6599c72527 [tube8] Extract categories and tags (Closes #10579) 2016-09-11 22:50:36 +07:00
6bb05b32a9 [pornhub] Extract categories and tags (closes #10499) 2016-09-11 19:22:51 +08:00
fea74acad8 [foxnews] Revert to old extractor names 2016-09-11 18:54:24 +08:00
f01115c933 [openload] Temporary fix (#10408) 2016-09-11 18:36:59 +08:00
2cdbc06a1f [foxnews] Support Fox News Articles (closes #10598) 2016-09-11 18:32:45 +08:00
2cb93afcd8 [viafree] Improve video id extraction (Closes #10615) 2016-09-11 14:59:14 +07:00
bfcda07a27 [abc:iview] Skip the test. They are removed soon 2016-09-11 04:06:00 +08:00
001a5fd3d7 [iwara] Fix extraction after relaunch
Closes #10462, closes #3215
2016-09-11 03:02:00 +08:00
1e35999c1e [tfo] Add new extractor 2016-09-10 19:43:31 +01:00
2512b17493 [lrt] Fix audio extraction (Closes #10566) 2016-09-11 01:27:20 +07:00
56c0ead4d3 [9now] Improve video data extraction (Closes #10561) 2016-09-11 00:42:13 +07:00
7324243750 [9now] Fix extraction 2016-09-11 00:16:29 +07:00
84a18e9b90 [polskieradio:category] Improve extraction 2016-09-10 22:01:49 +07:00
b29f842e0e [canalplus] Add support for c8.fr (Closes #10577) 2016-09-10 20:46:45 +07:00
f009fcac0d Merge branch 'master' of github.com:rg3/youtube-dl 2016-09-10 19:21:03 +07:00
6c3affcb18 [newgrounds] Fix uploader extraction
Closes #10584

Also change test URLs to HTTPS, as proposed by
@stepshal in #10593.

Closes #10593
2016-09-10 20:09:09 +08:00
1e19ff2984 Merge branch 'polskie-radio-programme' of https://github.com/JakubAdamWieczorek/youtube-dl 2016-09-10 00:42:36 +07:00
c6129feb7f [ketnet] Add extractor (Closes #10343) 2016-09-09 23:20:45 +07:00
bb5ebd4453 [canvas] Add support for een.be (Closes #10605) 2016-09-09 22:16:21 +07:00
cb9cbd84ed [extractors] add import for TeleQuebecIE 2016-09-08 22:55:27 +01:00
4d5726b0d7 [telequebec] Add new extractor(closes #1999) 2016-09-08 22:53:44 +01:00
4614ad7b59 [parliamentliveuk] fix extraction(closes #9137) 2016-09-08 20:46:12 +01:00
b717837190 release 2016.09.08 2016-09-08 23:46:14 +07:00
2abad67e52 [ChangeLog] Actualize 2016-09-08 23:32:16 +07:00
ad0e2b3359 [abcotvs] Add support for ABC Owned Television Stations 2016-09-08 23:15:58 +07:00
37720844f6 [jwplatform] Extract height from label 2016-09-08 22:53:20 +07:00
6cfcb8ac36 [tvnoe] Do not capture unused groups in _VALID_URL 2016-09-08 22:53:20 +07:00
7a979da8cb [yahoo] Look for Brightcove Legacy Studio embeds(closes #9345) 2016-09-08 16:44:22 +01:00
2fdc7b0e04 [viafree] PEP 8 2016-09-08 22:40:02 +07:00
010d034fca [videomore] Fix extraction (Closes #10592) 2016-09-08 22:38:49 +07:00
02e552886f Merge pull request #10596 from stepshal/r_prefix
Add missing r prefix for _VALID_URLs
2016-09-08 18:31:09 +08:00
25042f7372 Add missing r prefix for _VALID_URLs 2016-09-08 17:04:57 +07:00
3f612f0767 Fix _VALID_URLs further (#10594) 2016-09-08 17:39:29 +08:00
17bf6e71cc Merge pull request #10594 from stepshal/https_support
Add support for https for rest of the exctractors.
2016-09-08 17:28:46 +08:00
881f35479d Credit @xyb for miaopai extractor (#10556) 2016-09-08 17:22:43 +08:00
89f257d6e5 Add support for https for rest of the exctractors. 2016-09-08 13:52:22 +07:00
e78a5428b6 [foxgay] Fix extraction (closes #10480) 2016-09-08 02:01:09 +08:00
6656a82481 [rmcdecouverte] Add new extractor(closes #9709) 2016-09-07 17:33:22 +01:00
d7e794928d [tlc] fix query string parsing 2016-09-07 17:33:22 +01:00
9c27188988 Merge branch 'xyb-miaopai' 2016-09-08 00:31:06 +08:00
b84d311d53 [ChangeLog] Update for #10556 2016-09-08 00:29:55 +08:00
f87feb4b68 [miaopai] Coding style (#10556) 2016-09-08 00:28:33 +08:00
2841bdcebb Merge branch 'miaopai' of https://github.com/xyb/youtube-dl into xyb-miaopai 2016-09-08 00:08:02 +08:00
84b91dd4e3 [gamestar] Fix metadata extraction (closes #10479) 2016-09-07 23:07:50 +08:00
92c9c2a88b [moevideo] Skip another removed test (#10474) 2016-09-07 22:21:59 +08:00
9d54b02bae [puls4] fix extraction(closes #10583) 2016-09-07 14:43:20 +01:00
846d8b76a0 [cctv] Add new extractor(closes #8153) 2016-09-07 10:11:09 +01:00
aa3f9fe695 Explain why and why not to specify --hls-prefer-native
This has been asked at http://stackoverflow.com/questions/39357037/what-does-youtube-dl-option-hls-prefer-native-do-any-downside-adding-to-youtu
2016-09-07 10:38:59 +02:00
8258f4457c [lci] Add new extractor(closes #10573) 2016-09-06 20:47:42 +01:00
948cd5b72d [wat] extract dash formats 2016-09-06 20:44:45 +01:00
8d3737cda7 [polskieradio] Add support for downloading whole programmes.
This extends the Polskie Radio (the Polish national radio) extractor to
enable the user to download all the broadcasts of a single programme.
2016-09-06 21:34:44 +02:00
155bc674c4 [viafree] Improve video id detection (Closes #10569) 2016-09-07 00:41:31 +07:00
c33c962adf [trutv] Add new extractor(#10519) 2016-09-06 15:56:17 +01:00
bdcc046d12 [turner] use android secure hls host and catch token extraction errors 2016-09-06 15:53:03 +01:00
a493f10208 using _parse_html5_media_entries to parse video tag 2016-09-05 23:08:33 +08:00
f3eeaacb4e [nick] Add test for #10559 2016-09-05 21:42:41 +07:00
b4d6a85d60 [nick] Add support for nickelodeon.nl (Closes #10559) 2016-09-05 21:33:14 +07:00
0b36a96212 [abcotvs] extend _VALID_URL and add support for clips.abcotvs.com(closes #9551) 2016-09-05 13:41:21 +01:00
bc22a79694 Credit @mcepl for #10524 2016-09-05 16:44:06 +08:00
340e31ca74 Merge branch 'PeterDing-bilibili' 2016-09-05 13:55:07 +08:00
973dee491f [ChangeLog] Update for #10190 2016-09-05 13:54:35 +08:00
1f85029d82 [bilibili] Simplify 2016-09-05 13:53:58 +08:00
95be19d436 [miaopai] Add new extractor 2016-09-05 13:53:09 +08:00
95843da529 Merge branch 'bilibili' of https://github.com/PeterDing/youtube-dl into PeterDing-bilibili 2016-09-05 13:47:24 +08:00
abf2c79f95 Merge branch 'mcepl-tvnoe' 2016-09-05 13:39:51 +08:00
b49ad71ce1 [ChangeLog] Update for #10524 2016-09-05 13:38:55 +08:00
9127e1533d [tvnoe] PEP8 and coding style 2016-09-05 13:37:36 +08:00
78e762d23c Add new extractor for TV Noe (Czech Christian TV).
Fixes #10520
2016-09-04 19:06:40 +02:00
4809490108 release 2016.09.04.1 2016-09-04 20:58:28 +07:00
8112bfeaba [ChangeLog] Actualize 2016-09-04 20:57:18 +07:00
d9606d9b6c release 2016.09.04 2016-09-04 20:51:48 +07:00
433af6ad30 [theplatform] fix player regex(closes #10546) 2016-09-04 14:24:41 +01:00
feaa5ad787 [youtube:playlist] Extend _VALID_URL 2016-09-04 20:12:34 +07:00
100bd86a68 [rottentomatoes] delegate extraction to InternetVideoArchiveIE 2016-09-04 11:45:29 +01:00
0def758782 [internetvideoarchive] extract all formats 2016-09-04 11:45:29 +01:00
919cf1a62f [downloader/dash] Abort if the first segment fails
Closes #10497, Closes #10542
2016-09-04 17:32:29 +08:00
b29cd56591 [pornovoisines] Fix extraction (closes #10469) 2016-09-04 17:01:39 +08:00
622638512b [rottentomatoes] Fix extraction
Closes #10467
2016-09-04 16:25:59 +08:00
37c7490ac6 [espn] Extend _VALID_URL (Closes #10549) 2016-09-04 04:59:46 +07:00
091624f9da [vimple] Extend _VALID_URL (Closes #10547) 2016-09-04 03:39:13 +07:00
7e5dc339de [youtube:watchlater] Fix extraction (Closes #10544) 2016-09-04 00:29:01 +07:00
4a69fa04e0 [downloader/dash] Abort download immediately after giving up on some fragment 2016-09-03 17:51:48 +07:00
2e99cd30c3 [downloader/dash:hls] Report exact fragment error on retry 2016-09-03 17:51:48 +07:00
25afc2a783 [downloader/dash:hls] Respect --fragment-retries and --skip-unavailable-fragments (Closes #10165, closes #10448) 2016-09-03 17:51:48 +07:00
9603b66012 Introduce --skip-unavailable-fragments 2016-09-03 17:51:48 +07:00
45aab4d30b [youjizz] Fix extraction. The site has moved to HTML5
Closes #10437
2016-09-03 18:37:36 +08:00
ed2bfe93aa [fc2:embed] Add ie_key 2016-09-03 18:22:00 +08:00
cdc783510b [foxnews:insider] Add new extractor
Closes #10445
2016-09-03 18:16:19 +08:00
cf0efe9636 [fc2:embed] New extractor for Flash player URLs
Closes #10512
2016-09-03 17:25:03 +08:00
dedb177029 Fix parsing of HTML5 media elements
This fixes an error in _parse_html5_media_entries in case
an audio or video tag directly uses a src attribute insted
of <source> elements in it's body.
2016-09-03 16:09:35 +07:00
86c3bbbced release 2016.09.03 2016-09-03 01:46:41 +07:00
4b3a607658 [ChangeLog] Actualize 2016-09-03 01:45:17 +07:00
3a7d35b982 Credit @C4K3 for #10536 2016-09-03 01:42:33 +07:00
6496ccb413 [youtube] Add support for rental videos' previews (Closes #10532) 2016-09-03 01:17:15 +07:00
3fcce30289 [drtv] Update tests 2016-09-02 23:53:17 +07:00
c2b2c7e138 [utils] Add quicktime to mimetype2ext 2016-09-02 23:50:42 +07:00
dacb3a864a [youtube:playlist] Fallback to video extraction for video/playlist URLs when playlist is broken (Closes #10537) 2016-09-02 23:43:20 +07:00
6066d03db0 [drtv] Modernize and make more robust 2016-09-02 23:02:15 +07:00
6562d34a8c [utils] Improve mimetype2ext 2016-09-02 22:57:48 +07:00
5e9e3d0f6b [drtv] Add support for dr.dk/nyheder
It's the same video player, the only difference is that the video player
is loaded differently, and certain metadata (title and description) is
not available under dr.dk/mu, so make it by default get that from some
of the html meta tags.

Skip the dr.dk/tv test

dr.dk/tv videos are only available for between 7 and 90 days due to
Danish law, and in certain cases may be readded. Skip this test as it is
no longer available.
2016-09-02 22:20:36 +07:00
349fc5c705 [facebook:plugins:video] Add extractor (Closes #10530) 2016-09-02 21:13:50 +07:00
2c3e0af93e [go] Add new extractor 2016-09-02 09:53:04 +01:00
6150502e47 [adobepass] check for authz_token expiration(#10527) 2016-09-01 22:29:20 +01:00
b207d5ebd4 [curiositystream] don't cache auth token 2016-09-01 19:46:58 +01:00
4191779dcd [nytimes] improve extraction 2016-09-01 19:08:29 +01:00
f97ec8bcb9 [glide] Remove unused import 2016-09-01 23:46:58 +07:00
8276d3b87a [thestar] Fix extraction (Closes #10465) 2016-09-01 23:46:15 +07:00
af95ee94b4 [glide] Fix extraction (Closes #10478) 2016-09-01 23:38:49 +07:00
8fb6af6bba [exfm] Remove extractor (Closes #10482) 2016-09-01 23:32:28 +07:00
f6af0f888b [youporn] Fix categories and tags extraction (Closes #10521) 2016-09-01 23:15:01 +07:00
e816c9d158 [extractor/common] Simplify _extract_m3u8_formats 2016-09-01 22:18:16 +07:00
9250181f37 [extractor/common] Restore NAME usage from EXT-X-MEDIA tag for formats codes in _extract_m3u8_formats (Closes #10522) 2016-09-01 21:37:25 +07:00
f096ec2625 [curiositystream] Add new extractor 2016-09-01 13:37:09 +01:00
4c8ab6fd71 [thvideo] Remove extractor. Website down.
Closes #10464

According to a screenshot in http://tieba.baidu.com/p/4691302183,
thvideo.tv is shut down "temporarily". I see no clues that it will be up
again, so I remove it here.
2016-09-01 17:04:41 +08:00
05d4612947 [movingimage] Adapt to the new domain name and fix extraction
Closes #10466
2016-09-01 16:58:16 +08:00
746a695b36 [myvidster] Update _TESTS (closes #10473) 2016-09-01 16:42:35 +08:00
165c54e97d [southpark.cc.com:español] Skip geo-restricted _TESTS
Breaks https://travis-ci.org/rg3/youtube-dl/jobs/156728175
2016-09-01 16:28:03 +08:00
2896dd73bc [cbs] extract once formats(closes #10515) 2016-09-01 08:00:13 +01:00
f8fd510eb4 [limelight] skip ism manifests and reduce requests 2016-08-31 18:32:15 +01:00
7a3e849f6e [porncom] Extract categories and tags (Closes #10510) 2016-08-31 22:23:55 +07:00
196c6ba067 [facebook] Extract timestamp (Closes #10508) 2016-08-31 22:12:37 +07:00
165620e320 [yahoo] extract more and better formats 2016-08-30 21:49:28 +01:00
4fd350611c release 2016.08.31 2016-08-31 02:39:39 +07:00
263fef43de [ChangeLog] Actualize 2016-08-31 02:37:40 +07:00
a249ab83cb [pyvideo] Remove debugging code 2016-08-31 01:56:58 +07:00
f7043ef39c [soundcloud] Fix _VALID_URL clashes with sets (Closes #10505) 2016-08-31 01:56:15 +07:00
64fc49aba0 [bandcamp:album] Fix title extraction (Closes #10455) 2016-08-31 00:29:49 +07:00
245023a861 [pyvideo] Fix extraction (Closes #10468) 2016-08-30 23:51:18 +07:00
3c77a54d5d [turner] keep video id intact 2016-08-30 10:46:48 +01:00
da30a20a4d [turner,cnn] move a check for wrong timestamp to CNNIE 2016-08-29 19:26:53 +01:00
1fe48afea5 [cnn] update _TEST for CNNBlogsIE and CNNArticleIE(closes #10489) 2016-08-29 18:24:16 +01:00
42e05be867 [ctv] add support for (tsn,bnn,thecomedynetwork).ca websites(#10016) 2016-08-29 18:24:16 +01:00
fe45b0e060 [9c9media] fix multiple stacks extraction and extract more metadata(#10016) 2016-08-29 18:24:16 +01:00
a06e1498aa [kusi] Update test 2016-08-29 22:54:33 +07:00
5a80e7b43a [turner] Skip invalid subtitles' URLs 2016-08-29 22:44:15 +07:00
3fb2a23029 [adultswim] Extract video info from onlineOriginals (Closes #10492) 2016-08-29 22:40:35 +07:00
7be15d4097 [bilibili] Support episodes
[extractor/bilibili] add md5 for testing

[extractor/bilibili] remove unnecessary headers

[extractor/bilibili] correct _TESTS; find thumbnail for episode

[extractor/bilibili] [Fix] restore removed tests
2016-08-29 23:31:08 +08:00
cd10b3ea63 [turner] Extract all formats 2016-08-29 22:13:49 +07:00
547993dcd0 [turner] Fix subtitles extraction 2016-08-29 21:52:41 +07:00
6c9b71bc08 [downloader/external] Recommend --hls-prefer-native for SOCKS users
Related: #10490
2016-08-29 19:05:38 +08:00
93b8404599 [generic,vodplatform] improve embed regex 2016-08-29 07:57:20 +01:00
9ba1e1dcc0 [played] Remove extractor (Closes #10470) 2016-08-29 08:26:07 +07:00
b8079a40bc [turner] fix secure m3u8 formats downloading 2016-08-28 17:51:53 +01:00
5bc8a73af6 [cartoonnetwork] make extraction work for more videos in the website
some videos require `networkName=CN2` to be present in the feed url
2016-08-28 17:08:26 +01:00
b3eaeded12 [tbs] Add new extractor(#10222) 2016-08-28 16:51:09 +01:00
ec65b391cb [cartoonnetwork] Add new extractor(#10110) 2016-08-28 16:51:09 +01:00
2982514072 [turner,nba,cnn,adultswim] add base extractor to parse cvp feeds 2016-08-28 16:51:09 +01:00
98908bcf7c [openload] Update algorithm again (#10408) 2016-08-28 22:49:46 +08:00
04b32c8f96 [bilibili] Fix extraction (closes #10375)
Thanks @gdkchan for the algorithm
2016-08-28 22:06:31 +08:00
40eec6b15c [openload] Fix extraction (closes #10408)
Thanks to @yokrysty again!
2016-08-28 20:27:52 +08:00
39efc6e3e0 [generic] Update some _TESTS 2016-08-28 15:46:11 +08:00
1198fe14a1 release 2016.08.28 2016-08-28 07:24:08 +07:00
71e90766b5 [README.md] Fix typo in download archive FAQ entry 2016-08-28 07:09:03 +07:00
d7aae610f6 [ChangeLog] Actualize 2016-08-28 07:00:15 +07:00
92c27a0dbf [periscope:user] Fix extraction (Closes #10453) 2016-08-28 02:35:49 +07:00
d181cff685 Merge branch 'steven7851-patch-2' 2016-08-27 01:17:12 +08:00
3b4b82d4ce [douyutv] Simplify 2016-08-27 01:16:39 +08:00
545ef4f531 Merge branch 'patch-2' of https://github.com/steven7851/youtube-dl into steven7851-patch-2 2016-08-26 22:29:46 +08:00
906b87cf5f [crackle] Revert to template-based thumbnail extraction
To reduce to number of HTTP requests
2016-08-26 19:58:47 +08:00
b281aad2dc [douyutv] Use new api
use lapi for flv info, and html5 api for room info
#10153 #10318
2016-08-26 07:32:54 +08:00
6b18a24e6e [tnaflix] Fix extraction (Closes #10434) 2016-08-26 05:57:52 +07:00
c9de980106 Credit @Xender for nhk:vod (#10424) 2016-08-26 04:49:52 +07:00
f9b373afda [nhk:vod] Improve extraction (Closes #10424) 2016-08-26 04:48:40 +07:00
298a120ab7 [nhk] Add extractor for VoD. 2016-08-26 04:15:51 +07:00
e3faecde30 [trutube] Remove extractor (Closes #10438) 2016-08-26 03:43:13 +07:00
a0f071a50d [usanetwork] Add new extractor 2016-08-25 19:41:31 +01:00
20bad91d76 [downloader/external] Clarify that ffmpeg doesn't support SOCKS
Ref: #10304
2016-08-25 22:38:06 +08:00
b54a2da433 [crackle] Fix extraction and update _TESTS (closes #10333) 2016-08-25 22:22:31 +08:00
dc2c37f316 [spankbang] Fix description and uploader (closes #10339) 2016-08-25 20:47:35 +08:00
c1f62dd338 [README] Clean up grammar in --download-archive paragraph 2016-08-25 14:45:01 +02:00
5a3efcd27c [README.md] Add FAQ entry for download archive 2016-08-25 18:57:31 +07:00
4c8f9c2577 [README.md] Add comments in sample configuration for clarity 2016-08-25 18:27:15 +07:00
f26a298247 [README.md] Use en-US URL in cookies FAQ entry 2016-08-25 18:19:41 +07:00
ea01cdbf61 [README.md] Clarify how to export cookies from browser for cookies FAQ entry 2016-08-25 18:17:45 +07:00
6a76b53355 [README.md] Quote URL in streaming to player FAQ entry 2016-08-25 18:05:01 +07:00
d37708fc86 [YoutubeDL] check only for None Value in thumbnails sorting 2016-08-25 11:53:47 +01:00
5c13c28566 raise unexpected error when no stream found 2016-08-25 09:55:23 +01:00
f70e9229e6 [discoverygo] detect when video needs authentication(closes #10425) 2016-08-25 09:11:23 +01:00
30afe4aeb2 [cbc] Add support for watch.cbc.ca 2016-08-25 08:49:44 +01:00
75fa990dc6 [YoutubeDL] add fallback value for thumbnails values in thumbnails sorting 2016-08-25 08:49:44 +01:00
f39ffc5877 [common] extract formats from #EXT-X-MEDIA tags 2016-08-25 08:49:44 +01:00
07ea9c9b05 [downloader/hls] fill IV with zeros for IVs shorter than 16-octet 2016-08-25 08:49:44 +01:00
073ac1225f [utils] add ac-3 to the list of audio codecs in parse_codecs 2016-08-25 08:49:44 +01:00
0c6422cdd6 [README.md] Add FAQ entry for streaming to player 2016-08-25 07:34:55 +07:00
08773689f3 [kickstarter] Silent the warning for og:description
Closes #10415
2016-08-25 01:29:32 +08:00
0c75abbb7b [mtvservices:embedded] Use another endpoint to get feed URL
Closes #10363

In the original mtvservices:embedded test case, config.xml is still used
to get the feed URL. Some other examples, including test_Generic_40
(http://www.vulture.com/2016/06/new-key-peele-sketches-released.html),
and the video mentioned in #10363, use another endpoint to get the feed
URL. The 'index.html' approach works for the original test case, too. So
I didn't keep the old approach.
2016-08-24 23:58:22 +08:00
97653f81b2 [bilibili] Mark as broken
Bilibili now uses emscripten, which is very difficult for reverse
engineering. I don't expect it to be fixed in near future, so I mark
it as broken.

Ref: #10375
2016-08-24 21:28:00 +08:00
d38b27dd9b release 2016.08.24.1 2016-08-24 10:11:04 +07:00
6d94cbd2f4 [ChangeLog] Actualize 2016-08-24 10:07:06 +07:00
30317f4887 [pluralsight] Modernize and make more robust 2016-08-24 08:52:12 +07:00
8c3e35dd44 [pluralsight] Add support for subtitles (Closes #9681) 2016-08-24 08:41:52 +07:00
c86f51ee38 release 2016.08.24 2016-08-24 01:38:46 +07:00
6e52bbb413 [ChangeLog] Actualize 2016-08-24 01:36:27 +07:00
05bddcc512 [youtube] Fix authentication (2) (Closes #10392) 2016-08-24 01:29:50 +07:00
1212e9972f [youtube] Fix authentication (#10392) 2016-08-24 00:25:21 +07:00
ccb6570e9e [syfy,bravotv] restrict drupal settings regex 2016-08-23 17:31:35 +01:00
18b6216150 [openload] Fix extraction (closes #10408)
Thanks @yokrysty for the algorithm
2016-08-23 21:55:58 +08:00
fb009b7f53 [bravotv] correct clip info extraction and add support for adobe pass auth(closes #10407) 2016-08-23 10:29:52 +01:00
3083e4dc07 [eagleplatform] Improve detection of embedded videos (Closes #10409) 2016-08-23 07:22:14 +07:00
7367bdef23 [awaan] fix extraction, modernize, rename the extractors and add test for live stream 2016-08-22 23:10:06 +01:00
ad31642584 [nrk,abc:iview] use _extract_akamai_formats 2016-08-22 07:54:08 +01:00
c7c43a93ba [common] add helper method to extract akamai m3u8 and f4m formats 2016-08-22 07:49:34 +01:00
96229e5f95 [mtvservices:embedded] Update config URL
All starts from #10363. The test case in mtvservices:embedded uses
config.xml, while the video from #10363 and the test case in generic.py
is broken. Both uses index.html for fetching the feed URL.
2016-08-22 13:56:09 +08:00
55d119e2a1 [abc:iview] Add new extractor(closes #6148) 2016-08-22 00:07:17 +01:00
6d2679ee26 release 2016.08.22 2016-08-22 04:17:34 +07:00
afbab5688e [ChangeLog] Actualize 2016-08-22 04:15:46 +07:00
3d897cc791 [ivi] Fix episode number extraction 2016-08-22 03:34:27 +07:00
cf143c4d97 [ivi] Add support for 720p and 1080p 2016-08-22 03:31:33 +07:00
ad120ae1c5 [extractor/common] Change the default m3u8 protocol in HTML5
Helper functions should have consistent default values
2016-08-22 02:26:07 +08:00
d0fa172e5f [firsttv] keep a test videos with multiple formats 2016-08-21 19:13:43 +01:00
f97f9f71e5 Merge branch 'TRox1972-charlierose' 2016-08-22 02:11:43 +08:00
526656726b [charlierose] Simplify and improve 2016-08-22 02:06:47 +08:00
9b8c554ea7 [firsttv] fix extraction(closes #9249) 2016-08-21 17:56:25 +01:00
d13bfc07b7 Merge branch 'charlierose' of https://github.com/TRox1972/youtube-dl into TRox1972-charlierose 2016-08-22 00:48:35 +08:00
efe470e261 [twitch] Renew authentication 2016-08-21 22:45:50 +07:00
e3f6b56909 [twitch] Refactor API calls 2016-08-21 22:09:29 +07:00
b1e676fde8 [twitch] Modernize 2016-08-21 21:28:02 +07:00
92d4cfa358 [kaltura] Fallback ext calculation on caption's format 2016-08-21 21:01:01 +07:00
3d47ee0a9e [zingmp3] fix extraction and add support for video clips(closes #10041) 2016-08-21 14:09:48 +01:00
d164a0d41b [README.md] Add a format selection example using comma
Ref: #10399
2016-08-21 20:00:48 +08:00
db29af6d36 [charlierose] Add new extractor 2016-08-21 11:29:48 +02:00
2c6acdfd2d [kaltura] Add test for #10279 2016-08-21 08:37:01 +07:00
fddaa76a59 [kaltura] Assume ttml to be default subtitles' extension 2016-08-21 08:28:36 +07:00
a809446750 [kaltura] Add subtitles support when entry_id is unknown beforehand (Closes #10279) 2016-08-21 08:28:36 +07:00
d8f30a7e66 [kaltura] Remove unused code 2016-08-21 08:28:36 +07:00
5b1d85754e [YoutubeDL] Autocalculate ext when ext is None 2016-08-21 08:28:36 +07:00
e25586e471 [cultureunplugged] fix extraction(closes #10330) 2016-08-20 20:02:49 +01:00
292a2301bf [cnn] add support for money.cnn.com videos(closes #2797) 2016-08-20 19:00:25 +01:00
dabe15701b [cbs, cbsnews] fix extraction(fixes #10393) 2016-08-20 13:25:32 +01:00
4245f55880 [dotsub] Replace test (Closes #10386) 2016-08-20 06:18:20 +07:00
5b9d187cc6 [imdb] Improve title extraction and make thumbnail non-fatal 2016-08-20 04:50:39 +07:00
39e1c4f08c [litv] Support 'promo' URLs (closes #10385) 2016-08-20 00:52:37 +08:00
19f35402c5 [snotr] Fix extraction (closes #10338) 2016-08-20 00:18:22 +08:00
70852b47ca [utils] Recognize units with full names in parse_filename
Reference: https://en.wikipedia.org/wiki/Template:Quantities_of_bytes
2016-08-20 00:17:26 +08:00
a9a3b4a081 [miomio] Adapt to the new API and update _TESTS
The test case is from #9680
2016-08-20 00:08:23 +08:00
ecc90093f9 [vuclip] Adapt to the new API and update _TEST 2016-08-19 23:56:09 +08:00
520251c093 [extractor/common] Recognize m3u8 manifests in HTML5 multimedia tags 2016-08-19 23:53:47 +08:00
55af45fcab [radiobremen] Update _TEST (closes #10337) 2016-08-19 23:12:30 +08:00
b82232036a [n-tv.de] Fix extraction (closes #10331) 2016-08-19 20:39:28 +08:00
e4659b4547 [utils] Correct octal/hexadecimal number detection in js_to_json 2016-08-19 20:37:17 +08:00
9e5751b9fe [globo:article] Relax _VALID_URL and video id regex (Closes #10379) 2016-08-19 01:13:45 +07:00
bd1bcd3ea0 release 2016.08.19 2016-08-19 00:15:12 +07:00
93a63b36f1 [ChangeLog] Actualize 2016-08-19 00:13:24 +07:00
8b2dc4c328 [options] Remove output template description from --help
Same reasons as for --format
2016-08-18 23:59:13 +07:00
850837b67a [porncom] Add extractor (Closes #2251, closes #10251) 2016-08-18 23:52:41 +07:00
13585d7682 [utils] Recognize lowercase units in parse_filesize 2016-08-18 23:32:00 +07:00
fd3ec986a4 [generic] Fix dbtv test (Closes #10364) 2016-08-18 21:35:41 +07:00
b0d578ff7b [dbtv] Relax embed regex 2016-08-18 21:30:55 +07:00
b0c8f2e9c8 [DBTV:generic] Add support for embeds 2016-08-18 21:29:27 +07:00
51815886a9 [vk:wallpost] Fix audio extraction 2016-08-18 06:14:05 +07:00
08a42f9c74 [vk] Fix authentication on python3 2016-08-18 05:22:23 +07:00
e15ad9ef09 [keezmovies] PEP 8 2016-08-18 04:39:31 +07:00
4e9fee1015 [hgtvcom:show] Add extractor (Closes #10365) 2016-08-18 04:37:14 +07:00
7273e5849b [discoverygo] extend _VALID_URL to support other networks 2016-08-17 11:03:09 +01:00
b505e98784 [extremetube] Revert display_id 2016-08-17 07:02:13 +07:00
92cd9fd565 [keezmovies] Make display_id optional 2016-08-17 07:01:32 +07:00
b3d7dce429 release 2016.08.17 2016-08-17 06:21:21 +07:00
a44694ab4e [ChangeLog] Actualize 2016-08-17 06:19:22 +07:00
ab19b46b88 [extremetube] Modernize 2016-08-17 06:02:12 +07:00
8804f10e6b [tube8] Modernize 2016-08-17 05:46:45 +07:00
6be17c0870 [mofosex] Extract all formats and modernize (Closes #10335) 2016-08-17 05:45:49 +07:00
8652770bd2 [keezmovies] Improve and modernize 2016-08-17 05:44:46 +07:00
2a1321a272 [vbox7:generic] Add support for vbox7 embeds 2016-08-17 01:02:59 +07:00
9c0fa60bf3 [vbox7] Add support for embed URLs 2016-08-17 00:42:02 +07:00
502d87c546 [mtg] Improve view count extraction 2016-08-17 00:32:28 +07:00
b35b0d73d8 [viafree] Add extractor (Closes #10358) 2016-08-17 00:21:30 +07:00
6e7e4a6edf [mtg] Add support for viafree URLs (#10358) 2016-08-17 00:19:43 +07:00
53fef319f1 [fxnetworks] extend _VALID_URL to support simpsonsworld.com 2016-08-16 16:22:34 +01:00
2cabee2a7d [amcnetworks] fix typo 2016-08-16 16:22:34 +01:00
11f502fac1 [theplatform] extract subtitles with multiple formats from the metadata 2016-08-16 16:22:34 +01:00
98affc1a48 [xvideos] Fix test 2016-08-16 21:20:15 +07:00
70a2829fee [xvideos] Fix HLS extraction (Closes #10356) 2016-08-16 21:17:52 +07:00
837e56c8ee [amcnetworks] extract episode metadata 2016-08-16 14:49:32 +01:00
b5ddee8c77 [amcnetworks] Add new extractor 2016-08-16 13:44:01 +01:00
fb64adcbd3 [adobepass] PEP 8 2016-08-16 04:45:21 +07:00
4f640f2890 [bbc:playlist] Fix tests 2016-08-16 04:43:10 +07:00
254e64a20a [bbc:playlist] Add support for pagination (Closes #10349) 2016-08-16 04:36:23 +07:00
818ac213eb [adobepass] add IE suffix to the extractor and remove duplicate constant 2016-08-15 21:36:34 +01:00
cbef4d5c9f [fxnetworks] add test and check geo restriction 2016-08-15 17:10:45 +01:00
bf90c46790 [fxnetworks] Add new extractor(closes #9462) 2016-08-15 16:34:32 +01:00
69eb4d699f [cbsnews] Remove invalid tests. CBS Live videos gets deleted soon. 2016-08-15 20:29:22 +08:00
6d8ec8c3b7 [ChangeLog] Update for CBSLocal and related changes 2016-08-15 13:39:43 +08:00
760845ce99 [cbslocal] Adapt to SendtoNewsIE 2016-08-15 13:37:37 +08:00
5c2d087221 [sendtonews] Fix extraction 2016-08-15 13:31:08 +08:00
b6c4e36728 [jwplatform] Parse video_id from JWPlayer data
And remove a mysterious comma from 115c65793a
2016-08-15 13:29:01 +08:00
1a57b8c18c [zippcast] Remove extractor (Closes #10332)
ZippCast is shut down
2016-08-15 08:25:24 +07:00
24eb13b1c6 [uplynk,viceland] update tests and change uplynk extractors names 2016-08-14 22:45:43 +01:00
525e0316c0 [adobepass] fix check for pendingLogout errors 2016-08-14 21:25:43 +01:00
7e60ce9cf7 [adobepass] clear cache in case of pendingLogout errors 2016-08-14 21:24:33 +01:00
e811bcf8f8 [viceland] raise ExtractorError for errors other than HTTP 400 2016-08-14 20:13:35 +01:00
6103f59095 [viceland] remove outdated comment 2016-08-14 19:08:35 +01:00
9fa5789279 [viceland] fix info extraction(closes #8799) 2016-08-14 19:04:23 +01:00
d2ac04674d [viceland] Add new extractor(#8799) 2016-08-14 18:04:50 +01:00
1fd6e30988 [adobepass] create separate class for adobe pass authentication 2016-08-14 18:04:50 +01:00
884cdb6cd9 [life:embed] Improve extraction 2016-08-14 20:49:11 +07:00
9771b1f901 [theplatform] use _get_netrc_login_info and fix session expiration check(#10345) 2016-08-14 11:55:28 +01:00
2118fdd1a9 [common] add separate method for getting netrc ligin info 2016-08-14 11:55:28 +01:00
320d597c21 [vgtv] Detect geo restricted videos (#10348) 2016-08-14 16:25:14 +07:00
aaf44a2f47 [uplynk] Add new extractor 2016-08-13 22:53:41 +01:00
fafabc0712 Update ChangeLog for #10342
[skip ci]
2016-08-14 02:33:15 +08:00
409760a932 Merge pull request #10342 from muphil/patch-1
[xiami] bug fix for extractor xiami.py
2016-08-14 02:30:50 +08:00
phi
097eba019d bug fix for extractor xiami.py
Before applying this patch, when downloading resources from xiami.com, it crashes with these:
Traceback (most recent call last):
  File "/home/phi/.local/bin/youtube-dl", line 11, in <module>
    sys.exit(main())
  File "/home/phi/.local/lib/python3.5/site-packages/youtube_dl/__init__.py", line 433, in main
    _real_main(argv)
  File "/home/phi/.local/lib/python3.5/site-packages/youtube_dl/__init__.py", line 423, in _real_main
    retcode = ydl.download(all_urls)
  File "/home/phi/.local/lib/python3.5/site-packages/youtube_dl/YoutubeDL.py", line 1786, in download
    url, force_generic_extractor=self.params.get('force_generic_extractor', False))
  File "/home/phi/.local/lib/python3.5/site-packages/youtube_dl/YoutubeDL.py", line 691, in extract_info
    ie_result = ie.extract(url)
  File "/home/phi/.local/lib/python3.5/site-packages/youtube_dl/extractor/common.py", line 347, in extract
    return self._real_extract(url)
  File "/home/phi/.local/lib/python3.5/site-packages/youtube_dl/extractor/xiami.py", line 116, in _real_extract
    return self._extract_tracks(self._match_id(url))[0]
  File "/home/phi/.local/lib/python3.5/site-packages/youtube_dl/extractor/xiami.py", line 43, in _extract_tracks
    '%s/%s%s' % (self._API_BASE_URL, item_id, '/type/%s' % typ if typ else ''), item_id)
  File "/home/phi/.local/lib/python3.5/site-packages/youtube_dl/extractor/common.py", line 562, in _download_json
    json_string, video_id, transform_source=transform_source, fatal=fatal)
  File "/home/phi/.local/lib/python3.5/site-packages/youtube_dl/extractor/common.py", line 568, in _parse_json
    return json.loads(json_string)
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'NoneType'

This patch solves exactly this problem.
2016-08-14 02:18:59 +08:00
73a85620ee release 2016.08.13 2016-08-13 23:17:11 +07:00
a560f28c98 [ChangeLog] Actualize 2016-08-13 23:01:35 +07:00
5ec5461e1a [pbs] Clarify comment on http formats 2016-08-13 22:50:18 +07:00
542130a5d9 [pbs] Fix description extraction and update tests 2016-08-13 21:59:29 +07:00
82997dad57 [franceculture] Fix extraction (Closes #10324) 2016-08-13 21:00:34 +07:00
647a7bf5e8 [pornotube] Fix extraction (Closes #10322) 2016-08-13 20:49:16 +07:00
77afa008dd [4tube] Fix metadata extraction (Closes #10321) 2016-08-13 19:55:09 +07:00
db535435b3 [bigflix] Remove an invalid test
There's no video anymore
2016-08-13 18:02:11 +08:00
c2a453b461 [imgur] Fix width and height extraction (Closes #10325) 2016-08-13 16:46:07 +07:00
cd29eaab95 [vbox7] Remove unused imports 2016-08-13 16:45:34 +07:00
52aa7e7476 [test_verbose_output] Fix tests under Python 3 2016-08-13 17:36:14 +08:00
e97c55ee6a [expotv] Improve extraction and update test 2016-08-13 16:29:05 +07:00
acfccacad5 [downloader/external:curl] Clarify why CurlFD should not capture stderr 2016-08-13 10:26:02 +01:00
5f2c2b7936 [test_utils] add test for option with not str value 2016-08-13 09:54:12 +01:00
cb55908e51 [vbox7] Fix extraction (Closes #10309) 2016-08-13 15:47:20 +07:00
e581224843 [tapely] Remove extractor. It's shut down
Closes #10323
2016-08-13 16:32:07 +08:00
f50365e91c [pbs] add test for videos with undocumented http formats and remove unused import 2016-08-13 09:10:09 +01:00
c366f8d30a [24video] Add support for me and xxx TLDs 2016-08-13 14:47:51 +07:00
6a26c5f9d5 [muenchentv] Fix extraction (Closes #10313) 2016-08-13 14:28:44 +07:00
bd6fb007de [24video] Fix comment count extraction 2016-08-13 14:22:47 +07:00
b69b2ff736 [sunporno] Add support for embed URLs 2016-08-13 14:13:49 +07:00
794e5dcd7e [sunporno] Fix metadata extraction (Closes #10316) 2016-08-13 14:09:35 +07:00
f0d3669437 [hgtv] Add new extractor(closes #3999) 2016-08-12 18:05:49 +01:00
98e698f1ff [external/curl] respect more downloader options and display progress 2016-08-12 12:30:02 +01:00
3cddb8d6a7 [pbs] check all http formats and remove unnecessary request
- some of the quality that not reported in the documentation
are available(4500k, 6500k)
- the videoInfo request doesn't work for a long time
2016-08-12 08:38:06 +01:00
990d533ee4 [crunchyroll] Add support for HLS (Closes #10301) 2016-08-12 00:56:16 +07:00
b0081562d2 release 2016.08.12 2016-08-12 00:22:22 +07:00
fff37cfd4f [ChangeLog] Actualize 2016-08-12 00:18:28 +07:00
a3be69b7f0 [viu] Remove from extractors 2016-08-12 00:14:51 +07:00
0fd1b1624c [goldenmoustache] Remove extractor (Closes #10298)
Now uses dailymotion
2016-08-11 23:52:17 +07:00
367976d49f [drtuber] Improve title extraction 2016-08-11 23:47:52 +07:00
0aef0771f8 [drtuber] Make dislike count optional (Closes #10297) 2016-08-11 23:47:27 +07:00
0c070681c5 [chirbit] Fix extraction (Closes #10296) 2016-08-11 23:37:56 +07:00
30b25d382d [francetvinfo] Relax _VALID_URL 2016-08-11 21:42:55 +07:00
e5f878c205 [ChangeLog] Add change log for #10269
[skip ci]
2016-08-11 19:13:41 +08:00
e2e84aed7e Merge branch 'lkho-pr/#10268' 2016-08-11 19:09:18 +08:00
b1927f4e8a [YoutubeDL] Disable newline conversion when writing subtitles
By default io.open() convert all '\n' occurrences to '\r\n' when writing
files. If the content already contains '\r\n', it will be converted to
'\r\r\n', breaking some video players.
2016-08-11 19:04:23 +08:00
3b9323d96e Merge branch 'pr/#10268' of https://github.com/lkho/youtube-dl into lkho-pr/#10268 2016-08-11 19:03:08 +08:00
7f832413d6 Preserve line endings for downloaded subtitle files 2016-08-10 23:40:50 +08:00
7f2ed47595 [rtlnl] Relax _VALID_URL (Closes #10282) 2016-08-10 21:07:43 +07:00
c3fa77bdef [formula1] Relax _VALID_URL (Closes #10283) 2016-08-10 21:00:40 +07:00
57ce8a6d08 [wat] improve extraction(#10281)
add alternative method to extract http formats
works even if the video is geo-restricted or removed
from public access(most of the cases)
2016-08-10 14:20:28 +01:00
69d8eeeec5 [ctsnews] Fix extraction 2016-08-10 11:38:38 +08:00
81c13222c6 [utils] Recognize more formats in unified_timestamp
Used in CtsNews
2016-08-10 11:37:23 +08:00
b1ce2ba197 release 2016.08.10 2016-08-10 00:20:44 +07:00
5c8411e968 [ChangeLog] Actualize 2016-08-10 00:18:28 +07:00
cc9c8ce5df [devscripts/prepare_manpage] Fix description strings starting with dash (Closes #10273) 2016-08-09 22:24:58 +07:00
20ef4123b9 [uol] remove unused import 2016-08-09 15:13:15 +01:00
4e62d26aa2 [uol] Add new extractor(#4263) 2016-08-09 15:09:08 +01:00
b657816684 Credit @singh-pratyush96 for #10223 2016-08-09 04:04:45 +07:00
9778b3e7ee Credit @zvonicek for #10242 and #10253 2016-08-09 04:03:52 +07:00
25dd58ca6a [metadatafromtitle] Remove unused exception class 2016-08-09 04:01:05 +07:00
5e42f8a0ad Make --metadata-from-title non fatal
Output a warning if the metadata can't be parsed from the title (and don't write any metadata) instead of raising a critical error.
2016-08-09 03:56:22 +07:00
1ad6b891b2 Add more checks for --min/max-sleep-interval arguments and use more idiomatic naming 2016-08-09 03:47:56 +07:00
7aa589a5e1 Fix --min/max-sleep-interval wording 2016-08-09 03:46:52 +07:00
065bc35489 Add --max-sleep-interval (Closes #9930) 2016-08-09 03:32:42 +07:00
3a380766d1 [rbmaradio] Improve, simplify and extract all formats (Closes #10242) 2016-08-09 02:46:29 +07:00
affaea0688 [rbmaradio] Fixed extractor 2016-08-09 02:18:33 +07:00
77426a087b [sonyliv] Improve (Closes #10258) 2016-08-09 02:16:28 +07:00
8991844ea2 [sonyliv] Add new extractor 2016-08-09 02:09:13 +07:00
082395d0a0 [extractor/generic] Add proper default to _search_json_ld call 2016-08-08 22:48:33 +07:00
e8ed7354e6 [flipagram] Add proper default to _search_json_ld call 2016-08-08 22:46:19 +07:00
1e7f602e2a [condenast] Make _search_json_ld call non fatal 2016-08-08 22:45:49 +07:00
522f6c066d [bbc] Add proper default to _search_json_ld call 2016-08-08 22:44:36 +07:00
321b5e082a [extractor/common] Respect default in _search_json_ld 2016-08-08 22:36:18 +07:00
3711fa1eb2 Revert "[flipagram] Make _search_json_ld non fatal"
This reverts commit d34995a9e3.
2016-08-08 21:49:45 +07:00
395c74615c Revert "[extractor/generic] Make _search_json_ld non fatal"
This reverts commit 958849275f.
2016-08-08 21:49:27 +07:00
3dc240e8c6 [sohu] Update _TESTS (closes #10260) 2016-08-08 18:48:21 +08:00
a41a6c5094 [chaturbate] Skip the invalid test 2016-08-08 13:06:02 +08:00
d71207121d [biqle] Skip an invalid test 2016-08-08 12:59:55 +08:00
b1c6f21c74 [aparat] Fix extraction 2016-08-08 12:59:07 +08:00
412abb8760 [bilibili] Update _TESTS 2016-08-08 12:57:17 +08:00
f17d5f6d14 [features.aol.com] Fix _TESTS 2016-08-08 12:52:36 +08:00
6bb801cfaf [cwtv] extract http formats 2016-08-07 22:58:12 +01:00
de02d1f4e9 [rozhlas] Fix regexes and improve extraction (Closes #10253) 2016-08-08 04:58:02 +07:00
e1f93a0a76 [rozhlas] Add new extractor 2016-08-08 04:41:45 +07:00
d21a661bb4 [README.md] Update Options Link
The link references a bad anchor. The updated link now references the correct anchor.
2016-08-08 03:46:42 +07:00
b2bd968f4b [kuwo:singer] Fix extraction 2016-08-07 22:59:34 +08:00
4a01befb34 release 2016.08.07 2016-08-07 21:12:41 +07:00
845dfcdc40 [ChangeLog] Actualize 2016-08-07 21:10:48 +07:00
d92cb46305 [discoverygo] Add extractor (Closes #10245) 2016-08-07 20:57:05 +07:00
a8795327ca [utils] Add support TV Parental Guidelines ratings in parse_age_limit 2016-08-07 20:45:18 +07:00
d34995a9e3 [flipagram] Make _search_json_ld non fatal 2016-08-07 19:06:55 +07:00
958849275f [extractor/generic] Make _search_json_ld non fatal 2016-08-07 19:04:22 +07:00
998f094452 [bbc] Remove proxy from test 2016-08-07 18:13:05 +07:00
aaa42cf0cf [bbc] PEP 8 2016-08-07 18:05:13 +07:00
9fb64c04cd [bbc] Add support for morph embeds (Closes #10239) 2016-08-07 18:01:50 +07:00
f9622868e7 [bbc] preserve format_id backward compatibility 2016-08-07 11:14:15 +01:00
37768f9242 [common] correctly lower the preference of m3u8 master manifest format 2016-08-07 10:59:09 +01:00
a1aadd09a4 [tnaflixnetworkbase] Improve title extraction 2016-08-07 16:00:09 +07:00
b47a75017b [tnaflix] Fix metadata extraction (Closes #10249) 2016-08-07 16:00:03 +07:00
e37b54b140 [fox] fix theplatform release url query 2016-08-06 20:53:39 +01:00
c1decda58c [openload] Fix extraction (closes #9706) 2016-08-07 02:44:15 +08:00
d3f8e038fe [utils] Add decode_png for openload (#9706) 2016-08-07 02:42:58 +08:00
ad152e2d95 [bbc] fix test 2016-08-06 19:36:12 +01:00
b0af12154e [bbc] reduce requests and improve format_id 2016-08-06 19:24:59 +01:00
d16b3c6677 [common] extract partOfTVSeries info in json-ld 2016-08-06 18:58:38 +01:00
c57244cdb1 [common] lower the preference of m3u8 master manifest format 2016-08-06 18:55:05 +01:00
a7e5f27412 [bbc] improve extraction
- extract f4m and dash formats
- improve format sorting and listing
- improve extraction of articles with `otherSettings.playlist`
2016-08-06 18:48:09 +01:00
089a40955c [pokemon] improve _VALID_URL 2016-08-06 12:08:14 +01:00
d73ebac100 [pokemon] Add new extractor(closes #10093) 2016-08-06 11:18:14 +01:00
e563c0d73b [condenast] fallback to loader.js if video.js fail 2016-08-05 21:01:16 +01:00
491c42e690 release 2016.08.06 2016-08-06 01:23:48 +07:00
7f2339c617 [ChangeLog] Actualize 2016-08-06 01:19:47 +07:00
8122e79fef [gamekings] Remove remnants 2016-08-06 00:12:37 +07:00
fe3ad1d456 [adultswim] Remove superfluous md5 from test 2016-08-06 00:02:05 +07:00
038a5e1a65 [adultswim] Add support for trailers (Closes #10235) 2016-08-06 00:00:05 +07:00
84bc23b41b [archiveorg] PEP 8 2016-08-05 23:16:19 +07:00
46933a15d6 [extractor/common] Support root JSON-LD lists (Closes #10203) 2016-08-05 23:14:32 +07:00
3859ebeee6 [tvplay] Capture and output native error message 2016-08-05 22:50:42 +07:00
d50aca41f8 [archiveorg] improve format extraction(closes #10219) 2016-08-05 16:42:15 +01:00
0ca057b965 [jwplatform] add support for playlist extraction and relative urls and improve audio detection 2016-08-05 16:42:15 +01:00
5ca968d0a6 [tvplay] Extract series metadata 2016-08-05 22:37:38 +07:00
f0d31c624e [tvplay] Add support for subtitles (Closes #10194) 2016-08-05 22:17:32 +07:00
08c655906c [5min] fix _VALID_URL(closes #10228) 2016-08-05 10:22:33 +01:00
5a993e1692 [natgeo] fix tests(closes #10229) 2016-08-05 10:13:26 +01:00
a7d2953073 [extractors] add tvp:embed import 2016-08-05 10:11:59 +01:00
fdd0b8f8e0 [tvp] extract video id from the webpage(fixes #7799) 2016-08-05 09:44:15 +01:00
f65dc41b72 [naver] extract upload date 2016-08-05 08:12:25 +01:00
962250f7ea [cbslocal] Fix timestamp parsing (closes #10213) 2016-08-05 11:44:50 +08:00
7dc2a74e0a [utils] Fix unified_timestamp for formats parsed by parsedate_tz() 2016-08-05 11:41:55 +08:00
b02b960c6b [naver] improve extraction(closes #8096) 2016-08-04 21:42:22 +01:00
4f427c4be8 [condenast] improve extraction 2016-08-04 18:30:56 +01:00
8a00ea567b [natgeo:episodeguide] Do not shadow url from outer scope 2016-08-04 23:21:04 +07:00
8895be01fc [5min] fix _VALID_URL 2016-08-04 16:55:12 +01:00
52e7fcfeb7 [engadget] Relax _VALID_URL 2016-08-04 16:34:47 +01:00
2396062c74 [5min] delegate extraction to AolIE
recently the 5min SenseHandler request return
HTTP Error 503: Service Unavailable error
2016-08-04 16:21:27 +01:00
14704aeff6 [kaltura] remove debugging line 2016-08-04 14:54:34 +01:00
3c2c3af059 [extractors] change imports for national geographic extractors 2016-08-04 12:20:56 +01:00
1891ea2d76 [nationalgeographic] Add support for National Geographic Episode Guide 2016-08-04 12:18:10 +01:00
1094074c04 [kaltura] extract subtitles and reduce requests 2016-08-04 09:39:06 +01:00
217d5ae013 [vodplatform] Add new extractor 2016-08-04 09:39:06 +01:00
8b40854529 [common] lower proto_preference of rtsp formats
Most of the time the RtspFD fail to download videos but it report
success of the download with this output:
[mpv] 0 bytes
[download] 100% of 0.00B
2016-08-04 09:39:06 +01:00
6bb0fbf9fb Revert "[README.md] Use full paths for all configuration files (#8863)"
This reverts commit 899d2bea63.
2016-08-04 09:54:28 +08:00
8d3b226b83 [gamekings] Remove extractor
Now covered by generic jwplayer
2016-08-03 22:06:10 +07:00
42b7a5afe0 [limelight] extract http formats 2016-08-03 13:12:51 +01:00
899d2bea63 [README.md] Use full paths for all configuration files (#8863) 2016-08-03 11:15:27 +08:00
9cb0e65d7e [ntvru] Fix extraction 2016-08-02 22:56:48 +07:00
b070564efb [extractor/common] Support multiple properties in _og_search_property 2016-08-02 22:55:14 +07:00
ce28252c48 [options] Add test that checks that --password=secret is hidden in verbose output 2016-08-02 17:03:46 +02:00
3aa9a73554 [options] Hide --password=secret in verbose output 2016-08-02 17:03:26 +02:00
6a9b3b61ea [comedycentral] Re-add shortnames
In cc99d4f826, the shortname feature got deleted by accident. Re-add it as a separate IE.
2016-08-02 14:02:31 +02:00
45408eb075 release 2016.08.01 2016-08-01 22:59:23 +07:00
eafc66855d [ChangeLog] Add recent changes 2016-08-01 22:56:01 +07:00
e03d3e6453 [cwtv] Add support for cwtvpr.com (Closes #10196) 2016-08-01 22:51:01 +07:00
a70e45f80a [limelight] keep videos marked as previewStream
e382b953f0 (commitcomment-18472915)
2016-08-01 16:25:41 +01:00
697655a7c0 [safari] Relax url regexes (Closes #10202) 2016-08-01 21:48:48 +07:00
e382b953f0 [limelight] skip preview and drm protected videos 2016-08-01 00:33:30 +01:00
116e7e0d04 [bloomberg] Support BPlayer() players (closes #10187) 2016-07-31 14:47:19 +08:00
cf03e34ad3 [yandexmusic:track] Fix extraction (Closes #10193) 2016-07-31 07:56:18 +07:00
2903137292 release 2016.07.30 2016-07-30 14:45:07 +07:00
9361f2169c [ChangeLog] Make extractor improvements' descriptions more concrete 2016-07-30 14:43:28 +07:00
35aa6c538f Add ChangeLog 2016-07-30 12:33:09 +08:00
fa9f1d16b8 [dailymotion:playlist] Carry long line 2016-07-29 22:47:34 +07:00
485fedf6fd [dailymotion:playlist] Optimize download archive processing 2016-07-29 22:45:41 +07:00
da0baba5c8 [rtve] Fix extraction for some videos
For example http://www.rtve.es/alacarta/videos/documentos-tv/documentos-tv-descredito/3574098/.
2016-07-29 17:20:27 +02:00
bb9f3bfedf Revert "[rtve] Fix extraction (#10076)"
This reverts commit c39b2ed990.

Apparently outside of Spain using 'auth/resources' is required (#10097).
2016-07-29 17:14:04 +02:00
dbc0b39b91 [tv2] Improve extraction 2016-07-29 22:01:34 +07:00
481c5c5137 [tv2:article] Fix extraction (Closes #10188) 2016-07-29 21:43:17 +07:00
0cacae2807 [twitch:clips] Sort formats 2016-07-29 09:01:53 +07:00
d9d56deadf release 2016.07.28 2016-07-28 02:42:57 +07:00
74ba450a81 [twitch:clips] Fix extraction (Closes #9767) 2016-07-28 22:30:09 +07:00
db19df6ca0 [extractor/generic] Add test for #10179 2016-07-28 22:20:08 +07:00
fbdf8d15d1 [soundcloud] Add _extract_urls (#10179) 2016-07-28 22:16:05 +07:00
94aae01548 [extractor/generic] Extract all soundcloud embeds (Closes #10179) 2016-07-28 22:15:15 +07:00
39eef54cf0 [ard:mediathek] Skip unavailable test 2016-07-28 21:38:23 +07:00
05c8268c81 [shared] Modernize and make more robust 2016-07-27 23:39:02 +07:00
289a16b4f3 [shared] Respect redirect URL (Closes #10170) 2016-07-27 23:28:01 +07:00
7935926baa [devscripts/show-downloads-statistics] Add support for paging 2016-07-27 00:14:40 +07:00
dcbb07c35a release 2016.07.26.2 2016-07-26 23:56:53 +07:00
40090e8d51 [extractor/common] Improve is_suitable
In order to fix breakage introduced by a3aa814b77
2016-07-26 23:54:06 +07:00
3e050d51d4 [orf:oe1] Relax _VALID_URL 2016-07-26 23:14:04 +07:00
ced70c8640 [cbc] PEP 8 2016-07-26 23:08:08 +07:00
9a700deea4 [instagram] Remove duplicate field in test 2016-07-26 23:07:16 +07:00
dc35ba0eba [mgtv] Fix typo 2016-07-26 23:06:21 +07:00
88bd486b9a [cbc] Improve extraction for videos embedded with clipId 2016-07-26 22:58:50 +07:00
7f8b92e3cf [bigflix] Update tests 2016-07-26 21:44:53 +07:00
35f6e0ff36 [mtv.de] Skip 2 geo-restricted tests 2016-07-26 13:19:47 +08:00
326fa4e6e5 [generic] Skip an invalid test 2016-07-26 13:16:04 +08:00
c74299a72c [cmt] Detect unavailable videos and update _TESTS 2016-07-26 13:13:14 +08:00
10a1bb3a78 [mtv] Fix for videos with missing bitrates 2016-07-26 13:12:24 +08:00
4d3e543c73 Update extractors.py 2016-07-26 11:17:28 +08:00
05d1e7aaa9 [generic] Fix an MTV test and another test that breaks nosetests 2016-07-26 11:11:36 +08:00
a3aa814b77 Update _TESTS for MTV sites 2016-07-26 11:10:41 +08:00
5c32a77cad [nextmovie] Remove extractor
This domain name now redirects to mtv.com
2016-07-26 11:08:55 +08:00
14a28e705b [test/test_all_urls] Remove *.cc.com tests 2016-07-26 11:08:09 +08:00
cc99d4f826 [comedycentral] Remove IEs for *.cc.com except tosh.cc.com
All other subdomains now redirects to cc.com/* URLs
2016-07-26 11:06:50 +08:00
712c7530ff [mtv] Extract more metadata and more
1. Remove MTVIggyIE. All www.mtviggy.com URLs now redirects to
   www.mtv.com
2. Fix MTVDEIE
3. Return multiple URLs from _transform_rtmp_url. This is for
   tosh.cc.com
2016-07-26 11:03:43 +08:00
0a147785e8 [camdemy] Extract duration properly 2016-07-25 23:03:58 +07:00
59eaf69e33 [camdemy] Fix camdemy 2016-07-25 23:03:43 +07:00
e8be2943a7 [smotri] Modernize, make more robust and fix tests 2016-07-24 18:38:18 +07:00
8fdc538b46 release 2016.07.24 2016-07-24 11:39:50 +07:00
9513c1eb17 [tvp] Update dash format comment 2016-07-24 11:03:39 +07:00
ae6fff4e64 [onet] Enable dash formats 2016-07-24 10:43:05 +07:00
5a65668e25 [dcn] Enable dash formats 2016-07-24 10:35:55 +07:00
f75e6890db [telegraaf] Make hls non fatal 2016-07-24 10:29:26 +07:00
d9cb92c840 [telegraaf] Enable dash formats 2016-07-24 10:29:09 +07:00
94c04a3c79 [arkena] Enable dash formats 2016-07-24 10:28:11 +07:00
f094834857 [extractor/common] Add support for $ in SegmentTemplate in MPD manifests 2016-07-24 10:27:16 +07:00
111de00289 [DailyMail] Improve title and description extraction 2016-07-24 05:37:13 +07:00
b4a131e1a5 [facebook] Relax _VALID_URL (Closes #10151) 2016-07-24 04:36:49 +07:00
f1991ce928 [arkena] Skip dash formats 2016-07-23 18:07:55 +07:00
6548030a17 Credit @rvanbekkum for arkena (#8682) 2016-07-23 18:00:19 +07:00
3a8947650b [arkenaplay] Remove extractor 2016-07-23 17:57:55 +07:00
1979969f91 [extractor/generic] Add support for arkena embeds 2016-07-23 17:56:48 +07:00
0673741af3 [extractors] Add imports for arkena and lcp 2016-07-23 17:56:29 +07:00
c8e170b209 [lcp] Improve extraction 2016-07-23 17:56:11 +07:00
bbe1f3634a [arkena] Improve extraction (Closes #8682) 2016-07-23 17:55:54 +07:00
4671dd41b2 [arkena:lcp] Add extractors 2016-07-23 17:01:09 +07:00
f164b97123 [utils] Add another f4m mimetype to mimetype2ext 2016-07-23 16:48:59 +07:00
5275efe30d release 2016.07.22 2016-07-22 23:11:28 +07:00
b13647cf3c [eporner] Fix extraction (Closes #10139) 2016-07-22 23:04:13 +07:00
add7d2a0e2 [pornhub] Make error regex less ambiguous (Closes #10138) 2016-07-22 21:24:09 +07:00
e298d3a08c [youtube] Fix authentication (Closes #10140) 2016-07-22 21:05:39 +07:00
fd8c8c7dcd [youtube:shared] Relax _VALID_URL 2016-07-21 22:58:34 +07:00
9158af16cc [bbc.co.uk:iplayer:playlist] Add support for group URLs 2016-07-21 22:37:36 +07:00
c6668e4ad1 [bbc.co.uk:iplayer:playlist] Skip unavailable test 2016-07-21 22:34:55 +07:00
84e8cca48b [youjizz] Relax _VALID_URL (Closes #10131) 2016-07-20 22:41:13 +07:00
790b06b7d4 [odatv] Improve (Closes #9285) 2016-07-20 21:43:22 +07:00
740d7c49c2 [odatv] Add extractor 2016-07-20 21:42:05 +07:00
4e51ec5f57 [extractors] Add import for comedycentral.tv 2016-07-19 22:50:37 +07:00
05087d1b4c [bbc] Improve extraction from sxml playlists 2016-07-19 22:49:38 +07:00
a66a73ee90 [ard] Add test for rbb-online 2016-07-18 02:25:31 +07:00
8188b923db release 2016.07.17 2016-07-17 19:04:29 +07:00
d993a1354d [README.md] Make download URLs consistent 2016-07-17 18:58:47 +07:00
e8882e7043 [spike] Relax _VALID_URL and improve extraction (Closes #10106) 2016-07-17 18:34:25 +07:00
1056821799 [viki] Fix tests (Closes #10098) 2016-07-17 18:13:54 +07:00
890e6d3309 [viki] Lower m3u8 preference
http URLs are always provde the same or better quality
2016-07-17 18:12:03 +07:00
246080d378 [viki] Override m3u8 formats acodec 2016-07-17 18:10:16 +07:00
b1ea680270 Revert "[bbc] extract more and better qulities from Unified Streaming Platform m3u8 manifests"
This reverts commit 0385aa6199.
2016-07-17 17:29:36 +07:00
45550d1039 [comedycentraltv] Add extractor (Closes #10101) 2016-07-17 16:58:58 +07:00
7cdfc4c90f [mtvservices] Strip description 2016-07-17 16:56:39 +07:00
af21f56f98 [ard] Add support for rbb-online (Closes #10095) 2016-07-17 03:40:58 +07:00
1a8f0773b6 [streamable] Fix title extraction and improve (Closes #9122) 2016-07-17 02:01:00 +07:00
59cc5bd8bf [streamable] Add extractor 2016-07-17 01:35:09 +07:00
49bc16b95e [nintendo] Improve playlist extraction (Closes #9986) 2016-07-17 00:01:25 +07:00
a2f9ca1e67 [nintendo] Add extractor 2016-07-16 23:58:53 +07:00
371ddb14fe [extractor/generic] Change twitter:player embeds priority to lowest (Closes #10090) 2016-07-16 15:59:43 +07:00
998895dffa [cloudy] Drop videoraj.to
videoraj.ch is now a shoe-selling website, and videoraj.to domain name
is gone.
2016-07-16 15:37:54 +08:00
aadd3ce21f [cliphunter] Update _TESTS 2016-07-16 15:37:54 +08:00
ae7b846203 [cbsnews] Update _TESTS of CBSNewsLiveVideoIE 2016-07-16 15:37:54 +08:00
21ba7d0981 [cbc] Skip geo-restricted test case 2016-07-16 15:37:54 +08:00
691fbe7f98 release 2016.07.16 2016-07-16 02:20:00 +07:00
2e221ca3a8 [YoutubeDL] Fix incomplete formats check 2016-07-16 01:18:05 +07:00
317f7ab634 [YoutubeDL] Fix format selection with filters (Closes #10083) 2016-07-16 00:55:43 +07:00
23495d6a39 Revert "[ffmpeg] Fix embedding subtitles (#9063)"
This reverts commit ccff2c404d.

Fixes #10081.

The new approach breaks embedding subtitles into video-only or
audio-only files. FFMpeg provides a trick: add '?' after the argument of
'-map' so that a missing stream is ignored. For example:

opts = [
    '-map', '0:v?',
    '-c:v', 'copy',
    '-map', '0:a?',
    '-c:a', 'copy',
    # other options...
]

Unfortunately, such a format is not implemented in avconv, either.
I guess adding '-ignore_unknown' if self.basename == 'ffmpeg' is the
best solution. However, the example mentioned in #9063 no longer serves
problematic files, so I can't test it. I'll reopen #9063 and wait for
another example so that I can test '-ignore_unknown'.
2016-07-15 20:02:36 +08:00
224db034ab [syfy] fix extraction(closes #9087)(closes #3820)(closes #2388) 2016-07-14 23:59:47 +01:00
ad27649be3 [3qsdn] Restrict src JS regex 2016-07-15 03:36:50 +07:00
84571be645 [orf:tvthek] Remove test md5 2016-07-15 03:17:29 +07:00
7b0d333a7e Fix unit tests for m3u8 and RTSP extractors that require ffmpeg or mplayer 2016-07-15 03:06:23 +07:00
342f0c3682 [ninenow] correct test url 2016-07-14 14:19:18 +01:00
38e0f16a94 [ninenow] Add new extractor(closes #5181) 2016-07-14 14:16:11 +01:00
e910fe2fe4 [brightcove] skip ism manifests 2016-07-14 14:13:57 +01:00
233b58dec7 Add extractor for rtve.es/television (fixes #10076) 2016-07-13 21:02:34 +02:00
c39b2ed990 [rtve] Fix extraction (#10076)
For http://www.rtve.es/alacarta/videos/documentos-tv/documentos-tv-revolucion-del-movil/3069778/ using 'auth/resources' fails, and other URLs seem to work fine.
2016-07-13 20:23:27 +02:00
35ec86689c [bbc] extract only the original Unified Streaming Platform m3u8 manifests
0385aa6199 (commitcomment-18233275)
manifests with higher birate require more time to check formats
2016-07-13 18:01:14 +01:00
c485959034 release 2016.07.13 2016-07-13 23:58:01 +07:00
a0560d8ab8 [ellentv] Improve extraction (Closes #10067) 2016-07-13 22:42:53 +07:00
0385aa6199 [bbc] extract more and better qulities from Unified Streaming Platform m3u8 manifests 2016-07-13 15:58:24 +01:00
00f4764cb7 [common] extract vbr, abr and fps for Unified Streaming Platform m3u8 manifests 2016-07-13 15:58:24 +01:00
51c2cd0b83 [extractors] Add vk:wallpost extractor import 2016-07-13 21:53:23 +07:00
5f5a9d6158 [vk] Improve login 2016-07-13 21:52:52 +07:00
2d19fb5072 [vk:wallpost] Add extractor 2016-07-13 21:51:44 +07:00
9d865a1af6 [travis] Skip downloading srelay
SOCKS tests never run on Travis CI due to unknown reasons, and
downloading them broke some tests (e.g.
https://travis-ci.org/rg3/youtube-dl/builds/144306425)
2016-07-13 14:27:14 +08:00
41aa44259d [shahid] try to bypass geo restriction and extract more metadata(closes #10062) 2016-07-12 23:15:38 +01:00
381ff44756 [devscripts/generate-download] Remove MD5 and SHA1 2016-07-12 09:09:54 +02:00
7f29cf545a [youtube] Add YouTube Red paid video reference test (#10059) 2016-07-12 02:10:35 +07:00
7d1219f3e0 [tmz] delegate extraction to KalturaIE 2016-07-11 19:08:22 +01:00
f1b4af7d79 [beightcove:new] remove html tags from description 2016-07-11 19:06:50 +01:00
8a8590a617 [dbtv] delegate extraction to BrightcoveNewIE 2016-07-11 16:30:24 +01:00
4a7a5e41f7 [tvplay] improve extraction 2016-07-11 14:51:44 +01:00
2a49d01600 [playvid] Update _TESTS
Blocks https://travis-ci.org/rg3/youtube-dl/jobs/143809100
2016-07-11 15:15:28 +08:00
b99af8a51c [biobiochiletv] Fix extraction and update _TESTS 2016-07-11 13:23:57 +08:00
8e7020daef [rudo] Add new extractor
Used in biobiochile.tv
2016-07-11 13:19:25 +08:00
a26bcc61c1 release 2016.07.11 2016-07-11 03:17:12 +07:00
5c4dcf8172 [vidzi] Add support for embed URLs (Closes #10058) 2016-07-11 03:14:39 +07:00
e9fb6a4bbe [youtube] Relax TFA regexes 2016-07-11 03:08:38 +07:00
e2dbcaa1bf [vuclip] Fix extraction 2016-07-11 00:52:25 +08:00
ae01850165 [miomio] Fix _TESTS 2016-07-11 00:03:24 +08:00
c3baaedfc8 [miomio] Support new 'h5' player (closes #9605)
Depends on #8876
2016-07-10 23:46:48 +08:00
0b68de3cc1 Merge pull request #8876 from remitamine/html5_media
[extractor/common] add helper method to extract html5 media entries
2016-07-10 23:40:45 +08:00
39e9d524e5 Credit @nehalvpatel for roosterteeth (#9864) 2016-07-10 01:30:12 +07:00
865b087224 [roosterteeth] Improve (Closes #9864) 2016-07-10 01:30:12 +07:00
3121b25639 [roosterteeth] Add extractor 2016-07-10 01:30:12 +07:00
0286b85c79 release 2016.07.09.2 2016-07-09 22:22:24 +07:00
ab52bb5137 [animeondemand] Fix typo 2016-07-09 22:20:34 +07:00
61a98b8623 [lynda] Remove md5 from test (Closes #10047) 2016-07-09 21:29:11 +07:00
6daf34a045 [facebook] Fix typo and break when found video_data (Closes #10048) 2016-07-09 21:25:07 +07:00
c03adf90bd [generic] Add the test. Closes #1638 2016-07-09 14:39:01 +08:00
0ece114b7b [vimeo] Recognize non-standard embeds (#1638) 2016-07-09 14:38:27 +08:00
5b6a74856b Merge pull request #9288 from reyyed/issue#9063fix
[ffmpeg] Fix embedding subtitles (#9063)
2016-07-09 14:29:53 +08:00
ce43100a01 release 2016.07.09.1 2016-07-09 10:06:40 +07:00
8cc9b4016d [srmediathek] extend _VALID_URL(closes #9373) 2016-07-09 03:22:09 +01:00
31eeab9f41 [ard] fix f4m extraction and skip tests with 404 errors 2016-07-09 03:22:09 +01:00
9558dcec9c [youtube:user] Preserve user/c path segment 2016-07-09 08:37:19 +07:00
6e6b70d65f [extractor/generic] Properly comment out a test 2016-07-09 08:37:19 +07:00
d417fd88d0 release 2016.07.09 2016-07-09 07:16:47 +07:00
9e4f5dc1e9 [animeondemand] Pass num for episode based videos 2016-07-09 07:13:32 +07:00
1251565ee0 [options] Rollback old behavior for configuratio files' encoding
Until agreed with some solution
2016-07-09 07:12:52 +07:00
1f7258a367 [animeondemand] Add support for full length films (Closes #10031) 2016-07-09 06:57:04 +07:00
0af985069b [flipagram] Improve extraction (Closes #9898) 2016-07-09 03:31:17 +07:00
0de168f7ed [extractor/generic] Detect schema.org/VideoObject embeds 2016-07-09 03:29:07 +07:00
95b31e266b [extractor/common] Add expected_type in json ld routines 2016-07-09 03:28:04 +07:00
6b3a3098b5 [extractor/common] Extract more metadata for VideoObject in _json_ld 2016-07-09 03:27:11 +07:00
2de624fdd5 [extractor/common] Introduce filesize metafield for thumbnails 2016-07-09 03:24:36 +07:00
3fee7f636c [flipagram] Add extractor 2016-07-09 03:23:32 +07:00
89e2fff2b7 [mgtv] pass geo verification headers for api request 2016-07-08 20:18:25 +01:00
cedc70b292 [facebook] Fix invalid video being extracted (Closes #9851) 2016-07-09 00:28:07 +07:00
07d7689f2e [le] extract http formats 2016-07-08 15:35:20 +01:00
ae8cb5328d Merge branch 'JakubAdamWieczorek-polskie-radio' 2016-07-08 19:35:21 +08:00
2e32ac0b9a [polskieradio] Fix regex in _TESTS 2016-07-08 19:34:53 +08:00
672f01c370 Merge branch 'polskie-radio' of https://github.com/JakubAdamWieczorek/youtube-dl into JakubAdamWieczorek-polskie-radio 2016-07-08 19:33:28 +08:00
e2d616dd30 [polskieradio] Add thumbnails. 2016-07-08 13:23:00 +02:00
0ab7f4fe2b [nick] support nickjr.com (closes #7542) 2016-07-08 15:11:28 +08:00
29c4a07776 [lynda] Fix test 2016-07-08 03:33:53 +07:00
826e911e41 Merge branch 'master' of github.com:rg3/youtube-dl 2016-07-07 19:42:22 +02:00
30d22dae8e [options] Do not decode Unicode on Python 2.x
The configuration file contents are being returned as unicode now, so decoding them is no longer necessary.
(Run python2 with -3 to see the warning before this commit)
2016-07-07 19:41:00 +02:00
ec3518725b [compat] Fix test_cmdline_umlauts on Python 2.6
The original statement raises uncaught UnicodeWarning on Python 2.6
2016-07-07 22:30:58 +08:00
5f87d845eb [tweakers] fix info extraction(closes #9516) 2016-07-07 12:51:42 +01:00
571808a7aa document comments in configuration file (fixes #10024) 2016-07-07 12:12:21 +02:00
dfe5fa49ae [compat] Fix compat_shlex_split for non-ASCII input
Closes #9871
2016-07-07 17:37:29 +08:00
01a0c511eb [radiocanada] extract more formats 2016-07-07 03:46:12 +01:00
b3d30315ce Merge pull request #9597 from remitamine/toutv
[toutv] fix info extraction(closes #1792)(closes #2082)
2016-07-07 01:51:01 +01:00
882af14d7d [toutv] fix info extraction(closes #1792)(closes #2082) 2016-07-07 01:47:28 +01:00
47335a0efa [telecinco] fix info extraction 2016-07-06 23:09:13 +01:00
34bc2d9dfd release 2016.07.07 2016-07-07 01:54:29 +07:00
08c7af4afa [kamcord] Add extractor (Closes #10001) 2016-07-07 01:50:39 +07:00
f7291a0b7c [daum.net] Fix extraction for specific examples
Closes #9972
2016-07-07 01:26:14 +08:00
c65aa4e9e1 [brightcove:legacy] Support 'playlistTabs' and skip a dead test
Closes #9965
2016-07-07 01:13:37 +08:00
ad213a1d74 [francetv] Recognize more Dailymotion embedded videos
Closes #9955
2016-07-06 23:37:54 +08:00
43f1e4e41e [onet] Add MD5 checksum 2016-07-06 20:32:03 +08:00
54b0e909d5 [amp] Fix a typo 2016-07-06 20:10:47 +08:00
f8752b86ac [Onet,ClipRs] Add new extractor for onet.tv and use it for clip.rs
Closes #9950
2016-07-06 20:09:05 +08:00
84c237fb8a [utils] Add get_element_by_class
For #9950
2016-07-06 20:02:52 +08:00
ab49d7a9fa use mimetype2ext to determine manifest ext in multiple extractors 2016-07-06 09:11:46 +01:00
b4173f1551 [utils] add mimetypes to determine manifest ext(m3u8, f4m, mpd) 2016-07-06 09:06:28 +01:00
2817b99cf2 [metacafe] fix info extraction(closes #8539)(closes #3253) 2016-07-06 02:19:55 +01:00
001fffd004 [spiegel:article] update test(closes #10018) 2016-07-06 00:16:41 +01:00
0e94b4713d release 2016.07.06 2016-07-06 00:54:23 +07:00
a6d3b89feb [prosiebensat1] Make downloading urls JSON non fatal 2016-07-06 00:52:48 +07:00
6c26815d63 [onionstudios] fix info extraction 2016-07-05 18:05:07 +01:00
73c4ac2c95 [youtube:channel] Improve channel id extraction and detect unavailable channels (Closes #10009) 2016-07-05 23:30:44 +07:00
84f214d840 [prosiebensat1] extract all formats 2016-07-05 17:11:45 +01:00
e3f88be7a9 [rtvnh] extract all formats 2016-07-05 14:45:39 +01:00
31af3e35e0 [sandia] remove unused imports 2016-07-05 13:39:24 +01:00
94a5cff91d [sendia] fix info extraction 2016-07-05 13:37:46 +01:00
77082c7b9e [slideshare] fix description extraction 2016-07-05 12:01:04 +01:00
252a1f75d2 [spiegel] improve info extraction 2016-07-05 11:46:25 +01:00
5abf513cf8 [stitcher] fix episode config extraction 2016-07-05 10:44:16 +01:00
c6054e3201 [xuite] Support videos with already encoded media id 2016-07-05 14:26:42 +08:00
4080530624 [youtube:shared] Recognize the new 'shared' URLs
Closes #10007
2016-07-05 13:15:05 +08:00
c25f1a9b63 release 2016.07.05 2016-07-05 06:32:46 +07:00
dfaa86b75e [test_utils] add test for smuggling a smuggled url 2016-07-04 21:36:32 +01:00
d9163ae3b6 [kaltura] fix extraction error for videos from multiple kaltura servers 2016-07-04 21:34:27 +01:00
dafafe7cf1 [la7] extract more info from a kaltura custom server 2016-07-04 17:59:58 +01:00
81953d1ae5 [kaltura] add support videos stored on custom kaltura servers(closes #5557) 2016-07-04 17:59:58 +01:00
3a212ed62e [iqiyi] Skip an unstable MD5 checksum 2016-07-04 11:25:46 +08:00
195f084542 [pornhub] Detect private videos (Closes #9987) 2016-07-04 03:27:00 +07:00
aa7a455b2e [README.md] Clarify configuration file may not exist by default 2016-07-04 01:24:33 +07:00
6a4e659c93 [yahoo] Recognize brightcove embed (Closes #9995) 2016-07-03 23:00:36 +07:00
40f3666f6b [test/test_http] Update tests for 38cce791c7 2016-07-03 23:50:55 +08:00
dd801bbe18 [brightcove] improve error detection 2016-07-03 16:37:22 +01:00
38cce791c7 Rename --cn-verfication-proxy to --geo-verification-proxy
And deprecate the former one

Since commit f138873900, this option is
not limited to China websites, so rename it.
2016-07-03 23:29:56 +08:00
bf3ae6a543 [devscripts/show-downloads-statictics] Add script for displaying downloads statistics 2016-07-03 22:20:14 +07:00
bff98341d5 release 2016.07.03.1 2016-07-03 21:28:55 +07:00
2644e911be [iqiyi] Fix extraction
See https://github.com/soimort/you-get/issues/1211#issuecomment-229011559
2016-07-03 22:19:56 +08:00
a5f67895d3 [nationalgeographic] restore http formats
there was a misunderstanding about the reason of 403 response
the problem happen only when the user use aria2c as a downloader
a1f6f5c768 (commitcomment-18107559)
2016-07-03 14:10:25 +01:00
15e4b6b758 [rai] Support an alternative form of embedded relinker URL
Closes #8551
2016-07-03 19:52:11 +08:00
2b28b892d8 [rai] Support videos with embedded content item ID (#8551) 2016-07-03 19:52:11 +08:00
7507fc98cb [README.md] Fix somes typo in coding conventions section 2016-07-03 18:35:28 +07:00
477b7a8474 [downloader/f4m] Fix for Rai live streams 2016-07-03 19:26:39 +08:00
034a884957 [rai] Support direct relinker URLs (closes #8552) 2016-07-03 19:26:39 +08:00
64436cb1a4 [nationalgeographic] skip download for national geographic channel tests(closes #9991) 2016-07-03 10:43:36 +01:00
f138873900 [rai] Fix extraction and update _TESTS
Closes #8617
Closes #9157
Closes #9232
2016-07-03 15:49:35 +08:00
e793338c88 [buzzfeed] Detect Facebook embed and update _TESTS
Closes #5701
2016-07-03 14:12:02 +08:00
369bb06206 [facebook] Improve embed detection (#5701) 2016-07-03 14:11:29 +08:00
2cb31d288e [history:topic] Relax _VALID_URL 2016-07-03 13:01:04 +07:00
c723d1cd8d [README.md] Update some codebase links 2016-07-03 11:35:13 +07:00
1f55234057 Add PULL_REQUEST_TEMPLATE.md 2016-07-03 11:31:49 +07:00
04006fae8d [README.md] Start writing youtube-dl coding conventions 2016-07-03 11:31:07 +07:00
4cb13d0d6a [hrti] Don't redefine variable in list comprehension 2016-07-02 23:02:14 +02:00
a1f6f5c768 [nationalgeographic] add support Adobe Pass auth 2016-07-02 21:24:22 +01:00
05c7feec77 [aenetworks] add support Adobe Pass auth 2016-07-02 21:24:22 +01:00
bf83024826 [theplatform] add basic support for Adobe Pass 2016-07-02 21:24:22 +01:00
a0cfd82dda release 2016.07.03 2016-07-03 03:19:22 +07:00
1b734adb2d [xtube] Fix extraction (Closes #9953, closes #9961) 2016-07-03 03:17:35 +07:00
9b724d7277 [extractors] Add hrti:playlist import 2016-07-03 02:25:39 +07:00
c3a5dd3b5d Credit @atopuzov for hrti (#9482) 2016-07-03 02:22:59 +07:00
e3755a624b [hrti] Improve and add support for playlists (Closes #9482) 2016-07-03 02:22:14 +07:00
95cf60e826 [utils] Add PUTRequest 2016-07-03 02:21:32 +07:00
6b03e1e25d [HRTi] Implement extractor for Croatian Radiotelevision 2016-07-03 02:20:41 +07:00
712b0b5b70 [la7.it] Fix the extractor 2016-07-02 23:49:03 +08:00
6a424391d9 [facebook] Make embed detection stricter to prevent false-positives 2016-07-02 23:15:55 +08:00
dbf0157a26 [generic] Add MD5 checksums 2016-07-02 21:58:07 +08:00
7deef1ba67 [generic] Support Wordpress "YouTube Video Importer" plugin
Closes #9938
2016-07-02 21:58:07 +08:00
fd6ca38262 [facebook] Improve Facebook embedded detection
Related to #9938.

Another example comes from 9834872bf6.
2016-07-02 21:58:07 +08:00
bdafd88da0 [vk] Extend _VALID_URLs to support new domain (Closes #9981) 2016-07-02 16:43:19 +07:00
7a1e71575e release 2016.07.02 2016-07-02 02:47:42 +07:00
ac2d8f54d1 [vine] Remove superfluous whitespace 2016-07-02 02:45:00 +07:00
14ff6baa0e [fusion] Improve 2016-07-02 02:44:37 +07:00
bb08101ec4 [Fusion] Add new extractor 2016-07-02 02:37:28 +07:00
bc4b2d75ba [pornhub] Add support for thumbzilla (Closes #8696) 2016-07-02 02:11:07 +07:00
35fc3021ba [periscope] Add another fallback source 2016-07-02 01:35:57 +07:00
347227237b [periscope] fix playlist extraction (#9967)
The JSON response changed and the extractor needed to be updated in order to gather the video IDs.
2016-07-02 01:29:11 +07:00
564dc3c6e8 [vine] Fix extraction (Closes #9970) 2016-07-02 01:24:57 +07:00
9f4576a7eb [twitch] Update usher URL (Closes #9975) 2016-07-01 23:16:43 +07:00
f11315e8d4 release 2016.07.01 2016-07-01 03:59:57 +07:00
0c2ac64bb8 [sixplay] Rename preference key to quality in format dict 2016-07-01 03:57:59 +07:00
a9eede3913 [test/compat] compat_shlex_split: test with newlines 2016-07-01 03:30:35 +07:00
9e29ef13a3 [options] Accept quoted string across multiple lines (#9940)
Like:

    -f "
    bestvideo+bestaudio/
    best
    "
2016-07-01 03:30:31 +07:00
eaaaaec042 [pornhub] Add more tests with removed videos 2016-07-01 03:18:27 +07:00
3cb3b60064 [pornhub] Relax removed message regex (Closes #9964) 2016-07-01 03:14:23 +07:00
044e3d91b5 [Pornhub] Fix error detection 2016-07-01 02:59:50 +07:00
c9e538a3b1 [ctvnews] use orderedSet, increase the number of items for playlists and use smaller bin list for test 2016-06-30 19:52:32 +01:00
76dad392f5 [meta] Clarify the source of uppod st decryption algorithm 2016-06-30 18:27:57 +01:00
9617b557aa [ctv] Add new extractor(closes #4077) 2016-06-30 18:22:35 +01:00
bf4fa24414 [ctvnews] Add new extractor(closes #2156) 2016-06-30 18:22:35 +01:00
20361b4f25 [rds] extract 9c9media formats 2016-06-30 18:22:35 +01:00
05a0068a76 [9c9media] Add new extractor 2016-06-30 18:22:35 +01:00
66a42309fa release 2016.06.30 2016-06-30 23:56:55 +07:00
fd94e2671a [meta] Add support for pladform embeds 2016-06-30 23:20:44 +07:00
8ff6697861 [pladform] Improve embed detection 2016-06-30 23:19:29 +07:00
eafa643715 [meta] Make duration and description optional
For iframe URLs
2016-06-30 23:06:13 +07:00
049da7cb6c [meta] Extend _VALID_URL 2016-06-30 23:04:18 +07:00
7dbeee7e22 [generic] make twitter:player extraction non fatal 2016-06-30 14:11:55 +01:00
93ad6c6bfa [sixplay] Add new extractor(closes #2183) 2016-06-30 13:50:49 +01:00
329179073b [generic] add generic support for twitter:player embeds 2016-06-30 12:01:30 +01:00
4d86d2008e [urplay] fix typo and check with flake8 2016-06-30 11:30:42 +01:00
ab47b6e881 [theatlantic] Add new extractor(closes #6611) 2016-06-30 04:08:56 +01:00
df43389ade [skysports] Add new extractor(closes #7066) 2016-06-30 02:54:21 +01:00
397b305cfe [meta] Add new extractor(closes #8789) 2016-06-30 00:21:03 +01:00
e496fa50cd [urplay] Add new extractor(closes #9332) 2016-06-29 20:19:31 +01:00
06a96da15b [eagleplatform] Improve embed detection and extract in separate routine (Closes #9926) 2016-06-29 23:01:34 +07:00
70157c2c43 [aenetworks] add support for movie pages 2016-06-29 16:55:17 +01:00
c58ed8563d [aenetworks] extract history topic playlist title 2016-06-29 16:18:16 +01:00
4c7821227c [aenetworks:historytopic] fix topic video url 2016-06-29 16:03:32 +01:00
42362fdb5e [aenetworks] add support for show and season for A&E Network sites and History topics(closes #9816) 2016-06-29 15:49:17 +01:00
97124e572d [arte:playlist] Fix test 2016-06-28 22:39:53 +07:00
32616c14cc [vrt] extract all formats 2016-06-28 14:02:03 +01:00
8174d0fe95 release 2016.06.27 2016-06-27 23:09:39 +07:00
8704778d95 [pbs] Check manually constructed http links (Closes #9921) 2016-06-27 23:06:42 +07:00
c287f2bc60 [extractor/generic] Use _extract_url for kaltura embeds (Closes #9922) 2016-06-27 22:45:26 +07:00
9ea5c04c0d [kaltura] Add _extract_url with fixed regex 2016-06-27 22:44:17 +07:00
fd7a7498a4 [test_all_urls] PEP 8 and change wording 2016-06-27 22:11:45 +07:00
e3a6747d8f New test-case: extractor names are supposed to be unique
@dstftw explained in
https://github.com/rg3/youtube-dl/pull/9918#issuecomment-228625878 that
extractor names are supposed to be unique. @dstftw has fixed the two
offending extractors, and here I add a test to ensure this does not
happen in the future.
2016-06-27 22:09:29 +07:00
f41ffc00d1 [skynewsarabia:article] Clarify IE_NAME 2016-06-27 05:08:09 +07:00
81fda15369 [sr:mediathek] Clarify IE_NAME 2016-06-27 05:07:12 +07:00
427cd050a3 [extractor/generic] Improve kaltura embed detection (Closes #9911) 2016-06-27 04:11:53 +07:00
b0c200f1ec [msn] Add test URL with non-alphanumeric characters 2016-06-26 22:03:36 +07:00
92747e664a release 2016.06.26 2016-06-26 21:15:24 +07:00
f1f336322d [msn] Fix extraction (Closes #8960, closes #9542) 2016-06-26 21:10:05 +07:00
bf8dd79045 [extractor/common] Fix sorting with custom field preference 2016-06-26 21:09:07 +07:00
c6781156aa [MSN] add new extractor 2016-06-26 21:07:59 +07:00
59bbe4911a [extractor/common] add helper method to extract html5 media entries 2016-06-26 14:04:08 +01:00
4f3c5e0627 [utils] add helper function for parsing codecs 2016-06-26 14:03:58 +01:00
f484c5fa25 [vidbit] Improve (Closes #9759) 2016-06-26 16:59:28 +07:00
88d9f6c0c4 [utils] Add support for name list in _html_search_meta 2016-06-26 16:57:14 +07:00
3c9c088f9c [Vidbit] Add new extractor 2016-06-26 16:52:52 +07:00
fc3996bfe1 [iqiyi] Remove codes for debugging 2016-06-26 15:45:41 +08:00
5b6ad8630c [iqiyi] Partially fix IqiyiIE
Use the HTML5 API. Only low-resolution formats available

Related: #9839

Thanks @zhangn1985 for the overall algorithm (soimort/you-get#1224)
2016-06-26 15:18:32 +08:00
30105f4ac0 [le] Move urshift() to utils.py 2016-06-26 15:17:26 +08:00
1143535d76 [utils] Add urshift()
Used in IqiyiIE and LeIE
2016-06-26 15:16:49 +08:00
7d52c052ef [generic] Fix test_Generic_76
Broken: https://travis-ci.org/rg3/youtube-dl/jobs/140251658
2016-06-26 11:56:27 +08:00
a2406fce3c Fix misspelling 2016-06-26 01:28:55 +07:00
3b34ab538c [svtplay] Extend _VALID_URL (#9900) 2016-06-26 00:29:53 +07:00
ac782306f1 [iqiyi] Mark broken 2016-06-26 00:25:41 +07:00
0c00e889f3 Credit @JakubAdamWieczorek for #9813 2016-06-25 23:35:57 +07:00
ce96ed05f4 [polskieradio] Add test with video 2016-06-25 23:31:21 +07:00
0463b77a1f [polskieradio] Improve extraction (Closes #9813) 2016-06-25 23:19:18 +07:00
2d185706ea [polskieradio] Add support for Polskie Radio.
Polskie Radio is the main Polish state-funded radio broadcasting service.
2016-06-25 23:19:18 +07:00
b72b44318c [utils] Add strip_or_none 2016-06-25 23:19:18 +07:00
46f59e89ea [utils] Add unified_timestamp 2016-06-25 23:19:18 +07:00
b4241e308e release 2016.06.25 2016-06-25 03:03:20 +07:00
3d4b08dfc7 [setup.py] Add file version information and quotes consistency (Closes #9878) 2016-06-25 02:50:12 +07:00
be49068d65 [youtube] Fix and skip some tests 2016-06-24 22:47:19 +07:00
525cedb971 [youtube] Relax URL expansion in description 2016-06-24 22:37:13 +07:00
de3c7fe0d4 [youtube] Fix 141 format tests 2016-06-24 22:27:55 +07:00
896cc72750 [mixcloud] View count and like count may be absent
Closes #9874
2016-06-24 17:26:12 +08:00
c1ff6e1ad0 [vimeo:review] Fix extraction for password-protected videos
Closes #9853
2016-06-24 16:48:37 +08:00
fee70322d7 [appletrailers] correct thumbnail fallback 2016-06-23 19:03:34 +01:00
8065d6c55f [dcn] extend _VALID_URL for awaan.ae and extract all available formats 2016-06-23 17:22:15 +01:00
494172d2e5 [appletrailers] extract info from an alternative source if available(closes #8422)(closes #8422) 2016-06-23 15:49:42 +01:00
6e3c2047f8 [tvp] extract all formats and detect erros 2016-06-23 04:36:16 +01:00
011bd3221b release 2016.06.23.1 2016-06-23 09:42:56 +07:00
b46eabecd3 [jsinterp] Relax JS function regex (Closes #9863) 2016-06-23 09:41:34 +07:00
0437307a41 [nbc:nbcnews] improve extraction and add msnbc to the extractor 2016-06-23 01:36:19 +01:00
22b7ac13ef [tf1] fix wat id extraction(closes #9862) 2016-06-23 00:14:34 +01:00
96f88e91b7 release 2016.06.23 2016-06-23 04:29:34 +07:00
3331a4644d [vk] Remove unused import 2016-06-23 04:27:10 +07:00
adf1921dc1 [xnxx] Improve _VALID_URL (Closes #9858) 2016-06-23 04:26:49 +07:00
97674f0419 [xnxx] Replace test 2016-06-23 04:24:00 +07:00
rr-
73843ae8ac [xnxx] fix url regex
The pattern has changed from "video123412" to "video-o8xa19".
The changes maintain backwards compatibility with old-style URLs.
2016-06-23 04:19:55 +07:00
f2bb8c036a [vk] Modernize 2016-06-23 04:18:43 +07:00
75ca6bcee2 [vk] Workaround buggy new.vk.com Set-Cookie headers 2016-06-23 04:17:13 +07:00
089657ed1f [vimeo:album] Add paged example URL 2016-06-23 02:00:03 +07:00
b5eab86c24 [vimeo:album] Impove _VALID_URL 2016-06-23 01:56:58 +07:00
c8e3e0974b [vimeo:channel] Improve playlist extraction 2016-06-23 01:28:36 +07:00
dfc8f46e1c [vimeo:channel] Add video id to url_result
This will allow us to decide much faster that we don't want an already archived video,
and will allow having to download webpages for each video that has already been downloaded,
thus significantly speeding up the archival of channels that have no new content.
2016-06-23 01:26:27 +07:00
c143ddce5d [vimeo] Override original URL only when necessary 2016-06-23 00:51:36 +07:00
169d836feb lazy-extractors: Fix after commit 6e6b9f600f
The problem was in the following code:

    class ArteTVPlus7IE(ArteTVBaseIE):

        ...

        @classmethod
        def suitable(cls, url):
            return False if ArteTVPlaylistIE.suitable(url) else super(ArteTVPlus7IE, cls).suitable(url)

And its sublcasses like ArteTVCinemaIE.

Since in the lazy_extractors.py file ArteTVCinemaIE was not a subclass of ArteTVPlus7IE, super(ArteTVPlus7IE, cls) failed.

To fix it we have to make it a subclass. Since the order of _ALL_CLASSES is arbitrary we must sort them so that the base classes are defined first. We also must add base classes like YoutubeBaseInfoExtractor.
2016-06-22 19:20:50 +02:00
6ae938b295 [Vine] Extract view count 2016-06-22 23:57:35 +07:00
cf40fdf5c1 release 2016.06.22 2016-06-22 23:43:24 +07:00
23bdae0955 [svt] Various improvements
+ [svt:play] Add fallback path looking for video id and fix extraction for oppetarkiv
* [svt:base] Detect geo restriction
* [svt:base] Extract series related metadata
2016-06-22 23:36:07 +07:00
ca74c90bf5 Fix issue downloading facebook videos
youtube-dl expects the format items to be returned as a list,
but when there's only one item Facebook returns a dict instead,
this wraps the dict in a list if necessary
2016-06-22 12:52:15 +01:00
7cfc1e2a10 [gametrailers] Remove extractor
gametrailers closed (see http://www.polygon.com/2016/2/8/10944452/gametrailers-shuts-down-after-13-year-run)
2016-06-21 22:31:41 +07:00
1ac5705f62 [gamespot] extract all formats 2016-06-21 13:37:57 +01:00
e4f90ea0a7 [svt] Fix extraction for SVTPlay (closes #9809) 2016-06-21 17:55:53 +08:00
cdfc187cd5 [cbs] Remove unused import 2016-06-20 22:40:33 +07:00
feef925f49 [streamcloud] Capture error message (#9840) 2016-06-20 22:40:22 +07:00
19e2d1cdea release 2016.06.20 2016-06-20 20:50:01 +07:00
8369a4fe76 [downloader/hls] Simplify and carry long lines 2016-06-20 21:55:17 +07:00
1f749b6658 Revert "[jsinterp] Avoid double key lookup for setting new key"
This reverts commit 7c05097633.
2016-06-20 13:29:13 +02:00
819707920a [cbs] fix _VALID_URL 2016-06-19 23:55:19 +01:00
43518503a6 [cbs,cbsnews,cbssports] reduce requests while extracting all formats 2016-06-19 23:40:00 +01:00
5839d556e4 [theplatform] reduce requests for theplatform feed info extraction 2016-06-19 23:37:05 +01:00
6c83e583b3 [radiojavan] PEP8
E275 is added in pycodestyle 2.6

See https://github.com/PyCQA/pycodestyle/pull/491
2016-06-19 13:32:08 +08:00
6aeb64b673 Merge pull request #8201 from remitamine/hls-aes
[downloader/hls] Add support for AES-128 encrypted segments in hlsnative downloader
2016-06-19 13:25:08 +08:00
6cd64b6806 [foxsports] extract http formats 2016-06-19 05:45:48 +01:00
e154c65128 [downloader/hls] Add support for AES-128 encrypted segments in hlsnative downloader 2016-06-19 01:01:40 +01:00
a50fd6e026 release 2016.06.19.1 2016-06-19 03:57:14 +07:00
6a55bb66ee [vimeo] Fix rented videos (Closes #9830) 2016-06-19 03:56:01 +07:00
7c05097633 [jsinterp] Avoid double key lookup for setting new key
In order to add a new key to both __objects and __functions dicts on jsinterp.py, it is
necessary to first verify if a key was present and if not, create the key and
assign it to a value.

However, this can be done with a single step using dict setdefault method.
2016-06-19 03:29:45 +07:00
589568789f release 2016.06.19 2016-06-19 02:30:29 +07:00
7577d849a6 [r7] Fix extraction and add support for articles (Closes #9826) 2016-06-19 02:25:34 +07:00
cb23192bc4 [closertotruth] Update and improve (Closes #8680) 2016-06-19 00:35:29 +07:00
41c1023300 [closertotruth] Add extractor
Removed print statement from code.

Replaced two regex searches with the corret ones.

Removed some unnecessary semicolumns

fixed title extraction

refactored everything to search_regex

processed comments on commit 5650b0d, fixed feedback from flake8

Improved regexes and returns info dict now.

Added support for closertotruth interview URL

Added support for episodes page
2016-06-18 23:19:56 +07:00
90b6288cce [arte:+7] Simplify _VALID_URL 2016-06-18 22:23:48 +07:00
c1823c8ad9 [README.md] Remove 'small' from description (#9814) 2016-06-18 22:08:48 +07:00
d7c6c656c5 [arte:+7] Expand _VALID_URL (Closes #9820) 2016-06-18 21:42:17 +07:00
b0b128049a [extractors] Update references to sportschau (#9799) 2016-06-18 13:43:47 +08:00
e8f13f2637 [sportschau.de] Fix extraction and moved to its own file (closes #9799) 2016-06-18 13:42:58 +08:00
b5aad37f6b [ard] Remove SportschauIE, which is now based on WDR (#9799) 2016-06-18 13:42:39 +08:00
6d0d4fc26d [wdr] Add WDRBaseIE, for Sportschau (#9799) 2016-06-18 13:40:55 +08:00
0278aa443f [br] Skip invalid tests 2016-06-18 12:53:48 +08:00
1f35745758 [azubu] Don't fail on optional fields 2016-06-18 12:39:08 +08:00
573c35272f [bbc] Skip a geo-restricted test case 2016-06-18 12:35:55 +08:00
09e3f91e40 [arte] Update _TESTS and fix for pages with multiple YouTube videos
Some tests are from #6895 and #6613
2016-06-18 12:34:58 +08:00
1b6cf16be7 [aftonbladet] Fix extraction 2016-06-18 12:27:39 +08:00
26264cb056 [adobetv] Use embedded data in the webpage
Sometimes the HTML webpage is returned even with '?format=json'
2016-06-18 12:21:40 +08:00
a72df5f36f [mtvservices] Fix ext for RTMP streams 2016-06-18 12:19:06 +08:00
c878e635de [bet] Moved to MTVServices 2016-06-18 12:17:24 +08:00
0f47cc2e92 release 2016.06.18.1 2016-06-18 06:20:34 +07:00
5fc2757682 release 2016.06.18 2016-06-18 06:00:05 +07:00
e3944c2621 [pornhd] Add working test 2016-06-18 05:50:17 +07:00
667d96480b [pornhd] Detect removed videos and modernize 2016-06-18 05:42:20 +07:00
e6fe993c31 [pornhd] Improve formats extraction 2016-06-18 05:37:53 +07:00
d0d93f76ea [pornhd] Fix metadata extraction 2016-06-18 05:30:46 +07:00
20a6a154fe [mtv] Use compat_xpath and fix FutureWarning 2016-06-18 04:46:26 +07:00
f011876076 [nickde] Add extractor (Closes #9778) 2016-06-18 04:40:48 +07:00
6929569403 [mitele] Extract series metadata and make title more robust (Closes #9758) 2016-06-18 04:06:19 +07:00
eb451890da [carambatv] Add extractor (Closes #9815) 2016-06-18 03:04:14 +07:00
ded7511a70 [bbccouk] Add support for playlists (Closes #9812) 2016-06-17 23:42:52 +07:00
d2161cade5 release 2016.06.16 2016-06-16 22:40:55 +07:00
27e5fa8198 [cda] Fix extraction (Closes #9803) 2016-06-16 22:33:12 +07:00
efbd1eb51a [wimp] Fix extraction and update _TESTS 2016-06-16 12:27:21 +08:00
369ff75081 [jwplatform] Improved JWPlayer support 2016-06-16 12:26:45 +08:00
47212f7bcb [utils] Don't transform numbers not starting with a zero
Fix test_Viidea and maybe others
2016-06-16 11:00:54 +08:00
4c93ee8d14 [imdb] Improve _VALID_URL (Closes #9788) 2016-06-15 22:34:55 +07:00
8bc4dbb1af [wrzuta.pl] Detect error and update _TESTS 2016-06-14 11:14:59 +08:00
6c3760292c [pornhub] Improve title extraction (Closes #9777) 2016-06-14 04:57:59 +07:00
4cef70db6c [devscripts/release.sh] Add flag for gpg-sign commits 2016-06-14 03:16:56 +07:00
ff4af6ec59 [lynda] Remove superfluous _NETRC_MACHINE 2016-06-14 02:49:33 +07:00
d01fb21d4c release 2016.06.14 2016-06-14 02:19:42 +07:00
a4ea28eee6 Credit @venth for wrzuta:playlist (#9341) 2016-06-14 02:15:47 +07:00
bc2a871f3e Credit @dracony for rockstargames (#9737) 2016-06-14 02:15:09 +07:00
1759672eed [wrzuta:playlist] Improve and simplify (Closes #9341) 2016-06-14 02:13:54 +07:00
fea55ef4a9 [wrzuta.pl:playlist] Added playlist extraction from wrzuta.pl 2016-06-14 02:10:48 +07:00
16b6bd01d2 [rockstargames] Improve and add Youtube fallback (Closes #9737) 2016-06-14 01:11:24 +07:00
14d0f4e0f3 Added extractor for rockstargames.com 2016-06-14 01:09:35 +07:00
778f969447 [twitch:clips] Add extractor (Closes #9767) 2016-06-14 00:06:31 +07:00
79cd8b3d8a [README.md] Suggest checking extractor code under all Python versions 2016-06-13 10:04:04 +07:00
b4663f12b1 [README.md] Update links to info dict metafields 2016-06-13 07:16:35 +07:00
b50e02c1e4 [README.md] Update links to options available for YoutubeDL 2016-06-13 07:05:32 +07:00
33b72ce64e [xfileshare] Improve removed videos detection 2016-06-13 01:19:54 +07:00
cf2bf840ba [xfileshare] Fix test 2016-06-13 01:11:14 +07:00
bccdac6874 [xfileshare:xvidstage] Add support for videos with packed codes (Closes #4335) 2016-06-13 01:11:04 +07:00
e69f9f5d68 [downloader/external] Decode error string before writing to stderr 2016-06-12 16:45:07 +07:00
77a9a9c295 release 2016.06.12 2016-06-12 12:06:48 +07:00
84dcd1c4e4 [streamcloud] Detect removed videos (Closes #3768) 2016-06-12 11:08:39 +07:00
971e3b7520 [nrk:skole] Fix extraction 2016-06-12 07:20:37 +07:00
4e79011729 [nrktv] Fix tests 2016-06-12 06:57:04 +07:00
a936ac321c [README.md] Document using output template in batch files (Closes #9717) 2016-06-12 06:39:31 +07:00
98960c911c [instagram] Extract metadata from JSON 2016-06-12 06:06:04 +07:00
329ca3bef6 [utils] Add try_get
To reduce boilerplate when accessing JSON
2016-06-12 06:05:34 +07:00
2c3322e36e [youporn] Fix metadata extraction 2016-06-12 04:49:37 +07:00
80ae228b34 [matchtv] Modernize 2016-06-12 01:57:23 +07:00
6d28c408cf [viki] Do not use a fallback language for title in the first try
In test_Viki_3, 'titles' gives a Hebrew title.
2016-06-11 23:00:44 +08:00
c83b35d4aa [viki] Update _TESTS 2016-06-11 22:39:13 +08:00
94e5d6aedb [viki] Skip a geo-restricted test 2016-06-11 21:49:01 +08:00
531a74968c [vimeo] Fix extraction for VimeoReview videos 2016-06-11 21:35:08 +08:00
c5edd147d1 [generic] Remove an invalid test
Now handled by telewebion.py
2016-06-11 18:39:58 +08:00
856150d056 [telewebion] Add new extractor (closes #5135) 2016-06-11 18:39:58 +08:00
03ebea89b0 Merge pull request #9755 from vxbinaca/patch-2
[utils] Change Firefox 44 to 47
2016-06-11 17:38:45 +08:00
15d106787e [utils] Change Firefox 44 to 47
See commit title.
2016-06-11 05:36:31 -04:00
7aab3696dd [kuwo] Update _TESTS 2016-06-11 15:37:04 +08:00
47787efa2b [leeco] Recognize Le Sports URLs (fixes #9750) 2016-06-11 13:14:41 +08:00
4a420119a6 release 2016.06.11.3 2016-06-11 08:34:30 +07:00
33751818d3 release 2016.06.11.2 2016-06-11 08:28:51 +07:00
698f127c1a [setup.py] Add python 3.5 classifier 2016-06-11 06:14:22 +07:00
fe458b6596 [limelight] Extract ttml subtitles (Closes #9739) 2016-06-11 05:57:27 +07:00
21ac1a8ac3 [limelight] Fix typo 2016-06-11 05:52:50 +07:00
79027c0ea0 [limelight] Improve _VALID_URLs 2016-06-11 05:40:02 +07:00
4cad2929cd [limelight] Fix _VALID_URLs 2016-06-11 05:30:44 +07:00
62666af99f [indavideo] Fix formats' height (Closes #9744) 2016-06-11 05:13:05 +07:00
9ddc289f88 [README.md] Document missing playlist fields in output template 2016-06-11 04:59:47 +07:00
6626c214e1 release 2016.06.11.1 2016-06-11 03:00:08 +07:00
d845622b2e release 2016.06.11 2016-06-11 02:41:48 +07:00
1058f56e96 Merge pull request #9747 from TRox1972/lynda
[Lynda] Extract course description
2016-06-11 02:34:58 +07:00
0434358823 [Lynda] Extract course description 2016-06-10 19:17:58 +02:00
3841256c2c [lynda] Skip login if already logged in 2016-06-10 23:01:52 +07:00
bdf16f8140 [lynda] Add support for new authentication (Closes #9740) 2016-06-10 22:40:18 +07:00
836ab0c554 [compat] Import html5 entities correctly 2016-06-10 18:12:57 +08:00
6c0376fe4f [dw] Skip an invalid test
DW documentaries only last for one or two weeks. See #9475
2016-06-10 16:53:40 +08:00
1fa309da40 [generic] Update test_Generic_40
The original link now redirects to an YouTube user channel.
2016-06-10 16:39:31 +08:00
daa0df9e8b [youtube:user] Support another URL form
Such an URL comes from http://www.gametrailers.com/. This is originally
a test case in GenericIE, but now seems all GameTrailers videos are on
YouTube.
2016-06-10 16:37:12 +08:00
09728d5fbc [audiomack:album] Force video_id to be strings
Related: be6217b261
2016-06-10 16:11:28 +08:00
c16f8a4659 [voicerepublic] Force video_id to be strings
Related: be6217b261
2016-06-10 16:04:28 +08:00
a225238530 [vporn] Improve error detection and update _TESTS 2016-06-10 15:12:53 +08:00
55b2f099c0 [utils] Decode HTML5 entities
Used in test_Vporn_1. Also related to #9270
2016-06-10 15:11:55 +08:00
9631a94fb5 [compat] Add compat_html_entities_html5
Used in tset_Vporn_1. Also Related to #9270
2016-06-10 15:05:24 +08:00
cc4444662c [generic] Remove Vulture embed detection
Vulture.com videos now hosts on YouTube, Vimeo, MTV, NBC News or Hulu.
Here's an example of Hulu:
http://www.vulture.com/2016/06/kimmel-interviews-mariah-carey-in-a-bathtub.html
2016-06-10 13:40:57 +08:00
de3eb07ed6 [generic] Detect NBC News embeds 2016-06-10 13:32:59 +08:00
5de008e8c3 [nbcnews] Support embed widgets
Used in some Vulture videos
2016-06-10 13:31:55 +08:00
3e74b444e7 [vulture] Remove the extractor
The first 10 URLs in google search "site:http://video.vulture.com/video"
is dead. I guess Vulture does not host videos on their own anymore.
2016-06-10 13:13:59 +08:00
e1e0a10c56 [weibo] Remove the extractor
The Weibo weishipin (微視頻, tiny videos) service is dead and now all
videos are hosted on Sina videos, which is covered by sina.py
2016-06-10 13:01:22 +08:00
436214baf7 [xfileshare] Skip an invalid test 2016-06-10 12:31:06 +08:00
506d0e9693 [xuite] Skip the invalid test 2016-06-10 12:29:58 +08:00
55290788d3 [yahoo] Yahoo doesn't like region names in lower cases
Fix test_Yahoo_7
2016-06-10 12:28:56 +08:00
bc7e7adf51 [wdr] Subtitles are TTML 2016-06-10 00:22:41 +08:00
b0aebe702c [godtv] Relax _VALID_URL 2016-06-09 21:34:47 +07:00
416878f41f [godtv] Add more tests 2016-06-09 21:33:51 +07:00
c0fed3bda5 [godtv] Improve and add support for playlists (Closes #9608) 2016-06-09 21:29:41 +07:00
bb1e44cc8e [godtv] Add extractor
[GodTV] Improvements
2016-06-09 21:27:27 +07:00
21efee5f8b [openload] Relax _VALID_URL
[openload] added to _TESTS, removed escape
2016-06-09 20:46:54 +07:00
e2713d32f4 [openload] Fix extraction. Thanks @perron375 for the solution
Closes #9706
2016-06-09 19:00:13 +08:00
e21c26daf9 Merge pull request #9395 from pmrowla/afreecatv
[afreecatv] Add new extractor for afreecatv.com VODs
2016-06-09 17:20:16 +08:00
1594a4932f [wdr] Misc changes 2016-06-09 13:49:35 +08:00
6869d634c6 [wdr] Simplify extraction 2016-06-09 13:41:12 +08:00
50918c4ee0 [wdr] Support radio players (closes #6147) 2016-06-09 13:04:30 +08:00
6c33d24b46 [utils] Add audio/mpeg to mimetype2ext()
Used in WDR live radios (#6147)
2016-06-09 12:58:24 +08:00
be6217b261 [YoutubeDL] Force string conversion on non string video ids 2016-06-09 05:34:19 +07:00
9d51a0a9a1 [vessel] Make hls formats non fatal 2016-06-09 04:13:38 +07:00
39da509f67 [vessel] Extract DASH formats 2016-06-09 04:12:48 +07:00
a479b8f687 [vessel] Use native hls by default 2016-06-09 04:09:32 +07:00
48a5eabc48 [extractor/generic] Add support vessel embeds (Closes #7083) 2016-06-09 04:02:27 +07:00
11380753b5 [vessel] Add support for embed urls and improve extraction 2016-06-09 04:00:47 +07:00
411c590a1f [youku:show] Add new extractor 2016-06-08 23:45:46 +08:00
6da8d7de69 [twitter] Update _TESTS 2016-06-08 21:48:12 +08:00
c6308b3153 [twitter] Fix extraction for videos with HLS streams
Closes #9623
2016-06-08 21:28:10 +08:00
fc0a45fa41 [twitter] Detect suspended accounts and update _TESTS 2016-06-08 21:12:14 +08:00
e6e90515db [nbc] Add the test case from #9578
Closes #9578
2016-06-08 20:50:01 +08:00
22a0a95247 [theplatform] Some NBC videos require an additional cookie
Related: #9578
2016-06-08 20:47:39 +08:00
50ce1c331c [downloader/external] Add another env for proxies in ffmpeg/avconv
Related sources:
https://git.libav.org/?p=libav.git;a=blob;f=libavformat/http.c;h=8fe8d11e1edfdbb04a8726db2c49cfef3f572aac;hb=HEAD#l152
https://git.libav.org/?p=libav.git;a=blob;f=libavformat/tls.c;h=fab243e93e20034e88e619188c13a44a5d8ccdb9;hb=HEAD#l63
https://github.com/FFmpeg/FFmpeg/blob/f8e89d8/libavformat/http.c#L191
https://github.com/FFmpeg/FFmpeg/blob/f8e89d8/libavformat/tls.c#L92
2016-06-08 14:43:52 +08:00
7264e38591 [bilibili] Fix for videos without upload time (closes #9710) 2016-06-08 14:31:40 +08:00
33d9f3707c [thesixtyone] Relax _VALID_URL (Closes #9714) 2016-06-08 02:22:04 +07:00
a26a9d6239 [livestream:event] Ensure video id is string (Closes #9721) 2016-06-07 23:53:08 +07:00
a4a8201c02 [wdr] Update _TESTS 2016-06-08 00:25:51 +08:00
a6571f1073 [common] Fix <bootstrapInfo> detection in F4M manifests
Regression since 0a5685b26f
2016-06-08 00:19:33 +08:00
57b6e9652e [canal+] Add support for d17.tv 2016-06-07 22:32:08 +07:00
3d9b3605a3 [canal+] Update tests 2016-06-07 22:26:18 +07:00
74193838f7 [canal+] Improve extraction (Closes #9718) 2016-06-07 22:12:20 +07:00
fb94e260b5 Merge pull request #9720 from Kagami/vlive-new-statuses
[vlive] Acknowledge vlive+ streams statuses
2016-06-07 21:22:53 +07:00
345dec937f [vlive] Acknowledge vlive+ streams statuses
Same as common statuses just with "PRODUCT_" prefix:
PRODUCE_LIVE_END, PRODUCT_COMING_SOON, etc.
2016-06-07 17:12:13 +03:00
4315f74fa8 Merge remote-tracking branch 'Boris-de/wdrmaus_fix#8562' 2016-06-07 12:29:18 +02:00
e67f688025 [compat] Add 'compat_input' to __all__ 2016-06-05 23:16:08 +02:00
db59b37d0b [devscripts/create-github-release] Make full published releases by default 2016-06-06 03:02:11 +07:00
244fe977fe [options] Add --load-info-json alias for symmetry with --write-info-json 2016-06-06 02:52:58 +07:00
7b0d1c2859 [__init__] Use write_string instead of compat_string (Closes #9689) 2016-06-05 21:01:20 +07:00
21d0a8e48b Merge pull request #9702 from Eun/patch-1
curl: follow redirect
2016-06-05 17:43:26 +08:00
47f12ad3e3 curl: follow redirect 2016-06-05 11:04:55 +02:00
8f1aaa97a1 [README.md] Update pypi instructions 2016-06-05 11:19:44 +07:00
9d78524cbe Merge pull request #9697 from ryandesign/ryandesign-README-MacPorts
Update README.md to mention MacPorts
2016-06-05 11:09:20 +07:00
bc270284b5 Update README.md to mention MacPorts 2016-06-04 21:30:22 -05:00
c93b4eaceb git pushMerge branch 'master' of github.com:rg3/youtube-dl 2016-06-04 22:55:21 +02:00
71b9cb3107 extend FAQ (#9696) 2016-06-04 22:55:15 +02:00
633b444fd2 [downloader/hls] Correct comment on twitch vods 2016-06-05 03:31:10 +07:00
51c4d85ce7 [downloader/hls] PEP 8 2016-06-05 03:21:43 +07:00
631d4c87ee [twitch:vod] Use native hls 2016-06-05 03:19:44 +07:00
1e236d7e23 [downloader/hls] Do not rely on EXT-X-PLAYLIST-TYPE:EVENT 2016-06-05 03:16:05 +07:00
2c34735267 [youtube] Add itags 256 and 258 2016-06-05 01:44:13 +07:00
39b32571df [devscripts/release.sh] Release to GitHub 2016-06-05 00:48:33 +07:00
db56f281d9 [devscripts/create-github-release] Add script for releasing on GitHub
Yet only Basic authentication is supported either via .netrc or by manual input
2016-06-05 00:47:26 +07:00
e92b552a10 [devscripts/buildserver] Use compat_input from compat 2016-06-05 00:44:51 +07:00
1ae6c83bce [compat] Add compat_input 2016-06-05 00:43:55 +07:00
0fc832e1b2 [vidio] Improve (Closes #9562) 2016-06-04 16:48:24 +07:00
7def35712a [vidio] Add extractor (Closes #7195)
[Vidio] fix fallback value and wrap duration in int_or_none

[Vidio] don't use video_id for _html_search_regex()
2016-06-04 16:48:24 +07:00
cad88f96dc disable uploading to yt-dl.org for now 2016-06-04 11:42:52 +02:00
762d44c956 [channel9] Add support for rss links (Closes #9673) 2016-06-04 04:57:16 +07:00
4d8856d511 [loc] Extract direct download links 2016-06-04 00:26:03 +07:00
c917106be4 [loc] Extract subtites 2016-06-03 23:55:22 +07:00
76e9cd7f24 [loc] Add support for another URL schema and simplify 2016-06-03 23:43:34 +07:00
bf4c6a38e1 release 2016.06.03 2016-06-03 23:25:24 +07:00
7f3c3dfa52 [loc] Improve (Closes #9521) 2016-06-03 23:19:11 +07:00
9c3c447eb3 [loc] Add extractor (Closes #3188)
Added extractor of loc.gov, which closes #3188. I am not an experienced programmer, so I am sure I did a bunch of mistakes, but the extractor works (for me at least).

[LibraryOfCongress] don't use video_id for _search_regex()

[LibraryOfCongress] Improvements
2016-06-03 22:17:35 +07:00
ad73083ff0 [bilibili] Add _part%d suffixes back (closes #9660) 2016-06-02 19:29:27 +08:00
1e8b59243f Merge pull request #9669 from bzc6p/master
Added sanitization support for Hungarian letters Ő and Ű
2016-06-02 18:23:54 +08:00
c88270271e Added sanitization support for Hungarian letters Ő and Ű 2016-06-02 11:51:48 +02:00
b96f007eeb Added sanitization support for Hungarian letters Ő and Ű 2016-06-02 11:39:32 +02:00
9a4aec8b7e [utils] Use bytes-like objects as header values on Python 2 2016-06-02 15:00:49 +08:00
54fb199681 [test/test_http] Fix getsockname() on Jython 2016-06-02 15:00:49 +08:00
8c32e5dc32 [test/test_utils] Add test for #9588 2016-06-02 15:00:49 +08:00
0ea590076f [utils] Always decode Location header
escape_url is broken for bytes-like objects
2016-06-02 15:00:49 +08:00
4a684895c0 [seeker] Add new extractor(closes #9619) 2016-06-01 21:20:25 +01:00
f4e4aa9b6b [revision3:embed] Add new extractor 2016-06-01 21:20:25 +01:00
5e3856a2c5 release 2016.06.02 2016-06-02 01:19:57 +07:00
6e6b9f600f [arte] Add support for playlists and rework tests (Closes #9632) 2016-06-02 01:10:23 +07:00
6a1df4fb5f [spankwire] Add support for new URL format (Closes #9657) 2016-06-01 21:23:58 +07:00
dde1ce7c06 [tf1] Fix a regular expression (closes #9656)
This is a Python bug fixed in 2.7.6 [1]

[1] https://github.com/rg3/youtube-dl/issues/9656#issuecomment-222968594
2016-06-01 20:04:43 +08:00
811586ebcf [generic] Update the UDNEmbed test case 2016-06-01 19:23:44 +08:00
0ff3749bfe [udn] Fix m3u8 and f4m extraction as well as improve 2016-06-01 19:23:09 +08:00
28bab13348 [generic,viewlift] Move a test case to the specialized extractor 2016-06-01 19:18:01 +08:00
877032314f [generic] Improve Kaltura detection
Closes #4004
2016-06-01 18:37:34 +08:00
e7d85c4ef7 use /track/video/file to determine if video exists 2016-05-31 17:28:49 +09:00
8ec2b2c41c [options] Add --limit-rate alias for rate limiting option
Closes #9644
In order to follow regular --verb-noun pattern and better conformity with wget and curl
2016-05-30 21:48:35 +07:00
197a5da1d0 [yandexmusic] Improve captcha detection 2016-05-30 03:26:26 +07:00
abbb2938fa release 2016.05.30.2 2016-05-30 03:12:12 +07:00
f657b1a5f2 release 2016.05.30.1 2016-05-30 03:03:06 +07:00
86a52881c6 [travis] unsubscribe @phihag 2016-05-29 21:29:38 +02:00
8267423652 release 2016.05.30 2016-05-30 01:18:23 +07:00
917a3196f8 [README.md] Update c runtime dependency FAQ entry 2016-05-30 01:03:40 +07:00
56bd028a0f [devscripts/buildserver] Listen on all interfaces 2016-05-30 00:21:18 +07:00
681b923b5c [devscripts/release.sh] Allow passing buildserver address as cli option 2016-05-29 23:36:42 +07:00
9ed6d8c6c5 [youku] Extract resolution 2016-05-29 13:54:05 +08:00
f3fb420b82 [devscripts/release.sh] Check for wheel 2016-05-29 11:49:14 +06:00
165e3561e9 [devscripts/buildserver] Check Wow6432Node first when searching for python
This allows building releases from 64bit OS
2016-05-29 10:02:00 +06:00
27f17c0eab [Makefile] Fix youtube-dl.1 target
Now it accepts output filename as argument
2016-05-29 09:11:16 +06:00
44c8892369 [devscripts/prepare_manpage] Fix manpage generation on Windows 2016-05-29 09:06:10 +06:00
f574103d7c [buildserver] Fix buildserver and make python2 compatible 2016-05-29 09:03:17 +06:00
6d138e98e3 Merge pull request #9621 from venth/feature/ignored_intellij
ignored intellij related files
2016-05-29 03:10:29 +08:00
2a329110b9 ignored intellij related files 2016-05-28 20:27:18 +02:00
2bee7b25f3 [Makefile] Cleanup m4a files
[ci skip]
2016-05-29 01:59:09 +08:00
92cf872a48 [.gitignore] Ignore mp3 files
[ci skip]
2016-05-29 01:59:01 +08:00
6461f2b7ec [bilibili] Fix extraction, improve and cleanup 2016-05-29 01:26:00 +08:00
807cf7b07f [udemy] Fix authentication for localized layout (Closes #9594) 2016-05-28 21:18:24 +06:00
de7d76af52 [coub] Add another test 2016-05-27 23:38:17 +06:00
11c70deba7 [coub] Add extractor (Closes #9609) 2016-05-27 23:34:58 +06:00
f36532404d [vk] Remove superfluous code 2016-05-27 22:19:10 +06:00
77b8b4e696 [extractor/common] Borrow quality metadata from parent set-level manifest for f4m 2016-05-27 01:47:44 +06:00
2615fa7584 [downloader/f4m] Simply select format when it's the only one 2016-05-27 01:46:12 +06:00
3a686853e1 [WDR] fixed parsing of playlists 2016-05-26 20:54:51 +02:00
949fc42e00 [WDR] the other wdrmaus.de pages also changed to the new player 2016-05-26 20:54:51 +02:00
33a1ff7113 [WDR] extract jsonp-url by parsing data-extension of mediaLink 2016-05-26 20:54:51 +02:00
bec2c14f2c [WDR] add special handling if alt-url is a m3u8 2016-05-26 20:54:51 +02:00
37f972954d [WDR] use _download_json with a strip_jsonp 2016-05-26 20:54:51 +02:00
3874e6ea66 [WDR] use single quotes for strings 2016-05-26 20:54:51 +02:00
fac2af3c51 [common] Fix m3u8 extraction in f4m manifests 2016-05-27 01:41:27 +08:00
6f8cb24219 [tvp] Expand _VALID_URL and improve naming (Closes #9602) 2016-05-26 22:21:55 +06:00
448bb5f333 [common] Fix non-bootstrapped support in f4m 2016-05-27 00:03:48 +08:00
293c255688 [utils] Remove debugging codes 2016-05-26 22:54:16 +08:00
ac88d2316e [dw] Support documentaries (closes #9475) 2016-05-26 22:48:47 +08:00
5950cb1d6d [utils] Support a new form of date
Found in dw.com (#9475)
2016-05-26 22:44:00 +08:00
761052db92 [playwire] Add the test (closed #9531) 2016-05-26 21:57:06 +08:00
240b60453e [common] Support m3u8 in f4m manifests
Related: #9531
2016-05-26 21:55:43 +08:00
85b0fe7d64 [playwire] Use _extract_f4m_formats
Related: #9531
2016-05-26 21:43:35 +08:00
0a5685b26f [common] Support non-bootstraped streams in f4m manifests
Related: #9531
2016-05-26 21:41:47 +08:00
6f748df43f [eporner] Make test only_matching 2016-05-25 20:51:17 +06:00
b410cb83d4 Merge pull request #9595 from Kagami/vlive-site-update
[vlive] Address site update
2016-05-25 19:24:15 +08:00
da9d82840a Merge pull request #9600 from wankerer/master
[eporner] fix for the new URL layout
2016-05-25 18:52:55 +08:00
4ee0b8afdb [eporner] fix for the new URL layout
Recently eporner slightly changed the URL layout, the ID that used to be
digits only are now digits and letters, so youtube-dl falls back to
the generic extractor that doesn't work.

Fix the matching regex to allow letters in ID.

[v2: added a test case]
2016-05-24 15:57:36 -07:00
1de32771e1 [eyedotv] Add new extractor(closes #9582) 2016-05-24 20:10:12 +01:00
688c634b7d skip some tests to reduce test time 2016-05-24 16:44:11 +01:00
0d6ee97508 Credit @TRox1972 for tosh.cc (#9566) and localnews8 (#9539) 2016-05-24 21:42:47 +06:00
6b43132ce9 [xhamster] Update tests 2016-05-24 21:38:27 +06:00
a4690b3244 [xhamster] url regex fix for videos with empty title. 2016-05-24 21:35:43 +06:00
444417edb5 [radiocanada] Add new extractor(#4020) 2016-05-24 15:58:27 +01:00
277c7465f5 [ooyala] check manifest ext with determine_ext and update tests for related extractors 2016-05-24 11:24:29 +01:00
25bcd3550e [vlive] Address site update
Changes:
* Fix video params extraction
* Don't make status request since status info now available on the page
* Remove unneeded code
* Fix test
2016-05-24 12:54:28 +03:00
a4760d204f [ooyala] use api v2 to reduce requests for format extraction 2016-05-24 00:22:29 +01:00
e8593f346a [ooyala] extract subtitles 2016-05-23 23:58:16 +01:00
05b651e3a5 [washingtonpost] reduce requests for m3u8 manifests 2016-05-23 13:04:50 +01:00
42a7439717 [cbs] allow to pass content id to the extractor(closes #9589) 2016-05-23 09:31:37 +01:00
b1e9ebd080 [washingtonpost] remove unnecessary code 2016-05-23 02:30:12 +01:00
0c50eeb987 [reuters] Add new extractor 2016-05-23 02:27:31 +01:00
4b464a6a78 [washingtonpost] improve format extraction and add support for video pages extraction 2016-05-23 00:48:11 +01:00
5db9df622f [life:embed] Use native hls 2016-05-23 04:22:09 +06:00
5181759c0d [life] Update _VALID_URL 2016-05-23 04:00:08 +06:00
e54373204a [lifenews] Fix metadata extraction 2016-05-23 03:44:04 +06:00
102810ef04 [voxmedia] fix volume embed extraction 2016-05-22 20:37:35 +01:00
78d3b3e213 [generic] Improve Livestream detection (closes #2234) 2016-05-23 01:40:11 +08:00
7a46542f97 [livestream] Video IDs should always be strings (#2234) 2016-05-23 01:40:11 +08:00
eb7941e3e6 [compat] Fix for XML with <!DOCTYPE> in Python 2.7 and 3.2
Such XML documents cause DeprecationWarning if python is run
with `-W error`
2016-05-23 01:40:11 +08:00
db3b8b2103 [tf1] add support for more related web sites 2016-05-22 17:03:17 +01:00
c5f5155100 [wat] extract all formats 2016-05-22 17:03:17 +01:00
4a12077855 [genric] Eliminate duplicated video URLs (closes #6562) 2016-05-22 22:23:20 +08:00
a4a7c44bd3 [README.md] Document solution for extremely slow start on Windows 2016-05-22 15:04:51 +06:00
70346165fe [bandcamp] raise ExtractorError when track not streamable (#9465)
* [bandcamp] raise ExtractorError when track not streamable

* [bandcamp] update md5 for second test

* don't rely on json-data, but just check for 'file'

* don't rely on presence of 'file'
2016-05-22 14:15:39 +08:00
c776b99691 [README.md] Remove Windows updating trickery
Windows updating fixed in e9297256d4.
2016-05-22 10:14:02 +06:00
e9297256d4 [update] Fix youtube-dl.exe updating from arbitrary directory (Closes #2718) 2016-05-22 10:06:45 +06:00
e5871c672b [README.md] Clarify location for youtube-dl.exe even more
%USERPROFILE% not in %PATH% by default.
2016-05-22 09:36:07 +06:00
9b06b0fb92 [README.md] Clarify updating on Windows 2016-05-22 09:26:06 +06:00
4f3a25c2b4 [README.md] Fix typo 2016-05-22 09:00:08 +06:00
21a19aa94d [README.md] Clarify location for youtube-dl.exe 2016-05-22 08:59:28 +06:00
c6b9cf05e1 [utils] Do not fail on unknown date formats in unified_strdate 2016-05-22 08:28:41 +06:00
4d8819d249 [extractor/generic] Add support for theplatform embeds (Closes #8636, closes #9476) 2016-05-22 06:52:39 +06:00
898f4b49cc [theplatform] Add _extract_urls 2016-05-22 06:47:22 +06:00
0150a00f33 [cc] Add test for tosh.cc (Closes #9566) 2016-05-22 02:58:41 +06:00
c8831015f4 [ComedyCentral] Add support for tosh.cc.com and cc.com/video-clips 2016-05-22 02:55:10 +06:00
92d221ad48 [periscope] Update uploader_id (Closes #9565) 2016-05-22 02:39:15 +06:00
0db9a05f88 [periscope:user] Adapt to layout changes (Closes #9563) 2016-05-22 02:15:56 +06:00
e03b35b8f9 release 2016.05.21.2 2016-05-21 21:47:39 +02:00
d2fee3c99e release.sh: also check for python3 rsa module 2016-05-21 21:47:22 +02:00
598869afb1 release 2016.05.21.1 2016-05-21 21:27:00 +02:00
7e642e4fd6 release: check for pandoc
Abort releaseing if pandoc is missing.
(pandoc was not included in my essential app database, and thus missing on my new machine.)
2016-05-21 21:26:57 +02:00
c8cc3745fb release 2016.05.21 2016-05-21 21:18:59 +02:00
4c718d3c50 [rtve] Recognize 'filmoteca' URLs 2016-05-21 17:37:35 +02:00
115c65793a [jwplatform] Don't fail with RTMP URLs without mp4:, mp3: or flv: 2016-05-21 13:50:38 +08:00
661d46b28f [cbslocal] Add new extractor (closes #9522) 2016-05-21 13:40:45 +08:00
5ce3d5bd1b [sendtonews] Add new extractor
Used in CBSLocal. Part of #9522
2016-05-21 13:39:42 +08:00
612b5f403e [jwplatform] Improved m3u8 and rtmp support
Changes made for SendtoNewsIE. Part of #9522
2016-05-21 13:38:01 +08:00
9f54e692d2 [anvato] Add new extractor
Used in CBSLocal (#9522)
2016-05-21 13:18:29 +08:00
7b2fcbfd4e [common] Skip TYPE=CLOSED-CAPTIONS lines in m3u8 manifests
According to [1], valid values for TYPE are AUDIO, VIDEO, SUBTITLES
and CLOSED-CAPTIONS. Such a value is found in Anvato master playlists,
though I don't use _extract_m3u8_formats() in the end.

Part of #9522.

[1] https://tools.ietf.org/html/draft-pantos-http-live-streaming-19#section-4.3.4.1
2016-05-21 13:16:28 +08:00
16da9bbc29 [common] Add _m3u8_meta_format() template
For extractors who handle m3u8 manifests by themselves. (eg., AnvatoIE)

Part of #9522
2016-05-21 13:15:28 +08:00
c8602b2f9b [nrk] Unquote subtitles' URLs 2016-05-21 05:09:16 +06:00
b219f5e51b [brightcove:new] Improve error reporting 2016-05-21 00:59:06 +06:00
1846e9ade0 [localnews8] Fix extractor (Closes #9539) 2016-05-20 22:31:08 +06:00
6756602be6 [LocalNews8] add extractor (Closes #9200) 2016-05-20 22:10:13 +06:00
6c114b1210 [extractor/generic] Remove generic id and title from wistia extractionand update tests 2016-05-20 21:55:35 +06:00
7ded6545ed [extractor/generic] Add test for wistia standard embed 2016-05-20 21:43:36 +06:00
aa5957ac49 [extractor/generic] Add support for async wistia embeds (Closes #9549) 2016-05-20 21:33:31 +06:00
64413f7563 [cbc] fix extraction for flv only videos(fixes #5309) 2016-05-20 16:21:23 +01:00
45f160a43c [wistia] Improve hls support 2016-05-20 21:16:08 +06:00
36ca2c55db [wistia] Skip storyboard and improve extraction 2016-05-20 21:04:01 +06:00
f0c96af9cb [wistia] Add alias and modernize 2016-05-20 20:55:10 +06:00
31a70191e7 [cbc] Add the test case from #5156 2016-05-20 19:04:50 +08:00
ad96b4c8f5 [common] Extract audio formats in SMIL
Found in http://www.cbc.ca/player/play/2657631896

Closes #5156
2016-05-20 19:02:53 +08:00
043dc9d36f [cbc] Fix for old-styled URLs
The URL http://www.cbc.ca/player/News/ID/2672225049/ (#6342) redirects
to http://www.cbc.ca/player/play/2672224672, while youtube-dl wasn't
able to handle it correctly.
2016-05-20 18:39:54 +08:00
52f7c75cff [cbc] extract http formats and update tests 2016-05-20 06:58:46 +01:00
f6e588afc0 [24video] Fix description extraction 2016-05-20 08:53:04 +06:00
a001296703 [learnr] Add new extractor(closes #4284) 2016-05-19 18:18:03 +01:00
2cbd8c6781 Merge pull request #9537 from TRox1972/p1
[Makefile] delete thumbnails
2016-05-19 16:58:44 +08:00
8585dc4cdc [Makefile] delete thumbnails 2016-05-19 01:21:38 +02:00
dd81769c62 [ndtv] Fix extraction 2016-05-19 04:34:19 +06:00
46bc9b7d7c [utils] Allow None in remove_{start,end} 2016-05-19 04:31:30 +06:00
b78531a36a [formula1] Add new extractor(closes #3617) 2016-05-18 22:24:46 +01:00
11e6a0b641 [nfb] Modernize and extract subtitles 2016-05-18 00:25:15 +06:00
15cda1ef77 [nfb] Fix uploader extraction 2016-05-17 23:46:47 +06:00
055f0d3d06 [abcnews] Added a new extractor (closes #3992)
Related: #6108, #8664, #9459
2016-05-17 15:38:57 +08:00
cdd94c2eae [utils] Check for None values in SOCKS proxy
Originally reported at
https://github.com/rg3/youtube-dl/pull/9287#issuecomment-219617864
2016-05-17 14:38:15 +08:00
36755d9d69 release 2016.05.16 2016-05-16 17:25:47 +02:00
f7199423e5 [groupon] Add support for Youtube embeds (Closes #9508) 2016-05-16 00:30:13 +06:00
a0a81918f1 [collegehumor] Remove extractor
It now uses brightcove
2016-05-15 22:07:51 +06:00
5572d598a5 [hearthisat] Update the first test 2016-05-15 15:44:04 +08:00
cec9727c7f [hearthisat] Detect invalid download links (fixes #9440) 2016-05-15 15:35:31 +08:00
79298173c5 [utils] Fix getheader in urlhandle_detect_ext
Fixes #7049, related to #9440
2016-05-15 15:34:50 +08:00
69c9cc2716 [xvideos] Extract html5 player formats (Closes #9495) 2016-05-15 03:38:04 +06:00
ed56f26039 [extractor/common] Improve name extraction for m3u8 formats 2016-05-15 03:34:35 +06:00
6f41b2bcf1 [extractor/generic] Improve 3qsdn embeds support (Closes #9453) 2016-05-14 23:58:25 +06:00
cda6d47aad [utils] Simplify integer conversion in js_to_json 2016-05-14 23:41:57 +06:00
5d39176f6d [extractor/generic:3qsdn] Add support for embeds 2016-05-14 23:40:34 +06:00
5c86bfe70f [3qsdn] Add extractor 2016-05-14 23:35:03 +06:00
364cf465dd [test_utils] PEP 8 2016-05-14 20:46:33 +06:00
ca950f49e9 [ora] Revert extraction to regexes
It's less fragile than using js_to_json with ora js
2016-05-14 20:45:18 +06:00
89ac4a19e6 [utils] Process non-base 10 integers in js_to_json 2016-05-14 20:39:58 +06:00
640eea0a0c [ora] minimise fragile regex shenanigans; recognise unsafespeech.com URLs 2016-05-14 20:13:06 +06:00
bd1e484448 [utils] js_to_json: various improvements
now JS object literals like { /* " */ 0: ",]\xaa<\/p>", } will be correctly converted to JSON.
2016-05-14 20:12:39 +06:00
a834622b89 Merge pull request #9492 from jwilk/teamcoco
[teamcoco] Fix base64 regexp
2016-05-14 20:02:40 +08:00
707bb426b1 Merge pull request #9493 from jwilk/errno
Don't hardcode errno constant
2016-05-14 20:00:11 +08:00
66e7ace17a Don't hardcode errno constant
The value of ENOENT is architecture-dependent, so don't assume it's
always 2.
2016-05-14 13:41:41 +02:00
791ff52f75 [teamcoco] Fix base64 regexp 2016-05-14 13:19:54 +02:00
98d560f205 [test/test_socks] Skip SOCKS tests
They occasional trigger errors or blocks
(https://travis-ci.org/rg3/youtube-dl/jobs/130184883)
2016-05-14 18:48:36 +08:00
afcc317800 Merge pull request #9466 from TRox1972/patch-1
Update README.md
2016-05-14 17:03:04 +08:00
b5abf86148 [cinemassacre] Remove extractor (Closes #9457)
It now uses jwplatform
2016-05-14 04:53:14 +06:00
134c6ea856 [YoutubeDL] Sanitize url for url and url_transparent extraction results 2016-05-14 04:46:38 +06:00
0730be9022 [sina] fix extraction(fixes #1146) 2016-05-13 20:25:01 +01:00
96c2e3e909 [imdb] Improve extraction 2016-05-13 23:25:05 +06:00
f196508f7b [imdb] Relax _VALID_URL (Closes #9481) 2016-05-13 22:19:00 +06:00
cc1028aa6d [openload] Fix extraction (closes #9472) 2016-05-13 18:11:08 +08:00
ad55e10165 [brightcove] change the protocol for m3u8 formats to m3u8_native 2016-05-13 08:35:38 +01:00
18cf6381f6 [nrk] extract m3u8 formats 2016-05-13 08:05:28 +01:00
cdf32ff15d [extractors] add import for UstudioEmbedIE 2016-05-13 05:25:32 +01:00
99d79b8692 [ustudio] add support ustudio app/embed urls 2016-05-13 05:21:45 +01:00
b9e7bc55da [mgtv] extract http formats 2016-05-12 22:46:23 +01:00
d8d540cf0d [nrk] Rework extractor (Closes #9470) 2016-05-13 02:07:12 +06:00
0df79d552a [twitch:bookmarks] Remove extractor
Bookmarks no longer available
2016-05-13 00:14:30 +06:00
0db3a66162 [twitch] Skip dead tests 2016-05-12 23:57:52 +06:00
7581bfc958 [utils] Unquote crendentials passed to SOCKS proxies
Fixes #9450
2016-05-13 00:27:25 +08:00
f388f616c1 Update README.md 2016-05-12 16:48:12 +02:00
a3fa6024d6 [bloomberg] Fix test_Bloomberg
In this test case, sometimes HLS is the best format while sometimes HDS
is. To prevent occasional test failures, force HDS to be the best
format. In the past, testing against HDS formats causes the same error
as #9214, which is fixed as #9377 landed.
2016-05-12 20:08:42 +08:00
1b405bb47d [downloader/f4m] Tolerate truncate segments when testing
Replaces #9216

Fixes #9214 and test_Bloomberg partially
2016-05-12 20:02:36 +08:00
7e8ddca1bb [vevo] Delay the georestriction check to prevent false alerts
Fixes #9408
2016-05-12 19:56:58 +08:00
778a1ccca7 [utils] Add Œ and œ found in French to ACCENT_CHARS
Fixes #9463
2016-05-12 19:48:48 +08:00
4540515cb3 [iqiyi] Fix 1080P extraction (closes #9446) 2016-05-12 18:48:27 +08:00
e0741fd449 [__init__] Simplify colon presence check 2016-05-11 22:03:30 +06:00
e73b9c65e2 Bugfix: Allow colons in custom HTTP header values. 2016-05-11 21:59:24 +06:00
702ccf2dc0 [compat] Rename shlex_quote and remove unused subprocess_check_output 2016-05-10 16:00:21 +08:00
28b4f73620 release 2016.05.10 2016-05-10 09:08:08 +02:00
c2876afafe [test/test_socks] Use a different port range
Seems on Travis CI, ports in the original range are often used.
2016-05-10 14:51:38 +08:00
6ddb4888d2 [options] Update --proxy description for SOCKS proxies 2016-05-10 14:51:38 +08:00
fa5cb8d021 [socks] Remove a superfluous clause 2016-05-10 14:51:38 +08:00
e21f17fc86 [test/test_socks] Test with local SOCKS servers 2016-05-10 14:51:38 +08:00
edaa23f822 [compat] Rename struct_(un)pack to compat_struct_(un)pack 2016-05-10 14:51:38 +08:00
d5ae6bb501 [utils] Add rationale for register_socks_protocols 2016-05-10 14:51:38 +08:00
51fb4995a5 [utils] Register SOCKS protocols in urllib and support SOCKS4A 2016-05-10 14:51:38 +08:00
9e9cd7248d [socks] Eliminate magic constants and improve 2016-05-10 14:51:38 +08:00
72f3289ac4 [test/test_socks] Add tests for SOCKS proxies 2016-05-10 14:51:38 +08:00
71aff18809 [socks] Support SOCKS proxies 2016-05-10 14:51:38 +08:00
dab0daeeb0 [utils,compat] Move struct_pack and struct_unpack to compat.py 2016-05-10 14:51:38 +08:00
4350b74545 [socks] Add socks.py from @bluec0re's public domain implementation
https://gist.github.com/bluec0re/cafd3764412967417fd3
2016-05-10 14:49:25 +08:00
2937590e8b [downloader/hls] PEP 8 2016-05-09 22:16:33 +06:00
fad7bbec3a [test_compat] Remove unused import 2016-05-09 22:15:55 +06:00
e62d9c5caa [downloader/external] Call ffmpeg with with HTTP_PROXY env variable set (#9437) 2016-05-09 22:05:12 +06:00
20cfdcc910 [test_compat] Avoid None values for compat_setenv 2016-05-09 22:00:14 +06:00
1292638754 [test_compat] Use compat_setenv 2016-05-09 21:58:38 +06:00
fe40f9eef2 [compat] Add compat_setenv 2016-05-09 21:55:03 +06:00
6104cc2985 [downloader/hls] Add event media playlists to unsupported features of hlsnative 2016-05-09 20:55:37 +06:00
c15c47d19b [downloader/hls] Remove EXT-X-MEDIA-SEQUENCE from unsupported features for hlsnative 2016-05-09 20:45:03 +06:00
965fefdcd8 Credit @sleep-walker for #9431 2016-05-09 20:38:33 +06:00
3951e7eb93 [ceskatelevize] Simplify, restore bonus video test and skip georestricted test (Closes #9431) 2016-05-09 20:37:20 +06:00
f1f6f5aa5e [ceskatelevize] Add support for live streams
Live streams has no playlist title, use title of the stream containing
TV channel name. Internal m3u8 handler doesn't seem to handle well
continuous streams. Add test for live stream. Remove no longer
reachable test.
2016-05-09 18:58:15 +06:00
eb785b856f Merge pull request #9358 from dstftw/hls-native-to-ffmpeg-delegation
[downloader/hls] Delegate extraction to ffmpeg when unsupported features detected
2016-05-08 22:07:55 +00:00
c52f4efaee [mva] Improve _VALID_URLs 2016-05-08 20:10:20 +06:00
f23a92a0ce [mva] Add extractor (Closes #6667) 2016-05-08 20:02:54 +06:00
3b01a9fbb6 [litv] Add new extractor
LiTV is a streaming platform providing free and paid legal contents in
Taiwan.
2016-05-08 14:34:38 +08:00
93fdb14177 don't use selection by attribute 2016-05-08 10:33:17 +09:00
370d4eb8ad use stricter file selector
in case of empty in case of empty ./track/video/file entries
2016-05-08 10:02:48 +09:00
3452c3a27c update tests 2016-05-08 10:02:19 +09:00
9c072d38c6 [arte] Improve language preference (Closes #9401, closes #9162) 2016-05-08 06:52:42 +06:00
81f35fee2f fix extractors.py import order 2016-05-08 08:57:16 +09:00
0fdbe3146c use dict.get in case upload_date does not exist 2016-05-08 08:56:22 +09:00
3e169233da Expanduser for more options with input files 2016-05-08 04:36:57 +06:00
f5436c5d9e [downloader/external] Add temp fix ffmpeg m3u8 downloads (Closes #9394) 2016-05-08 02:29:26 +06:00
5c24873a9e Credit @inondle for #9400 2016-05-08 02:04:34 +06:00
00c21c225d Credit @kdeldycke for #9430 2016-05-08 00:11:44 +06:00
d013b26719 Merge pull request #9430 from kdeldycke/batch_file_home_expansion
Expand user's home in batch file path.
2016-05-07 18:09:51 +00:00
e2eca6f65e Expand user's home in batch file path. 2016-05-07 20:03:25 +02:00
a0904c5d80 [telegraaf] Fix extractor (closes #9318) 2016-05-08 00:56:31 +08:00
cb1fa58813 [flickr] Extract uploader URL (Closes #9426) 2016-05-07 20:15:40 +06:00
3fd6332c05 [flickr] extract license field(closes #9425) 2016-05-07 15:13:14 +01:00
401d147893 Merge pull request #9400 from inondle/master
[liveleak] Adds support for thumbnails and updates tests
2016-05-06 19:23:31 +00:00
e2ee97dcd5 [liveleak] Adds support for thumbnails, updates tests 2016-05-06 12:05:37 -07:00
f745403b5b [vevo] Revert videoplayer.vevo.com to api.vevo.com 2016-05-06 23:37:17 +06:00
3e80e6f40d [vevo] Allow request to api.vevo.com to fail (Closes #9417)
I don't know whether this it's tempopary or api has just gone
2016-05-06 23:35:58 +06:00
25cb7a0eeb [youtube] Allow empty attribute values in description regex 2016-05-06 22:11:18 +06:00
abc97b5eda [utils] Allow empty attribute values in get_element_by_attribute (Closes #9415) 2016-05-06 22:07:30 +06:00
04e88ca2ca [vk] improve extraction(fixes #7976) 2016-05-06 15:02:40 +01:00
8d93c21466 add multi_video test case 2016-05-06 12:08:43 +09:00
1dbfd78754 fix multi_video part naming, add upload_date field 2016-05-06 12:07:29 +09:00
22e35adefd use url instead of single formats entry 2016-05-06 10:41:30 +09:00
6f59aa934b [periscope:user] Add new extractor for user pages
Closes #9388
2016-05-06 02:14:39 +08:00
109db8ea64 Merge pull request #9367 from codesparkle/master
Feature: --restrict-filenames: replace accented characters by their unaccented counterpart instead of "_"
2016-05-06 01:44:03 +08:00
833b644fff use xpath_text 2016-05-06 01:24:02 +09:00
915620fd68 [redtube] PEP 8 2016-05-05 21:34:06 +06:00
ac12e888f9 [redtube] Extract all formats, duration, upload date and view count (Closes #9397) 2016-05-05 21:02:54 +06:00
b1c6a5bac8 [Makefile] Remove more media files in make clean 2016-05-05 20:50:39 +08:00
7d08f6073d [kuwo:category] Update test 2016-05-05 20:20:26 +08:00
758a059241 [dailymail] Add new extractor(closes #2667) 2016-05-05 13:13:22 +01:00
4f8c56eb4e [fczenit] Fix extraction and update test
Closes #9359
2016-05-05 17:55:37 +08:00
57cf9b7f06 [afreecatv] Add new extractor for afreecatv.com VODs 2016-05-05 03:59:23 +09:00
9da526aae7 [yandexmusic:playlist] Update test 2016-05-04 23:18:48 +06:00
75b81df3af [udemy] Modernize 2016-05-04 23:14:12 +06:00
aabdc83d6e [udemy] Fix course enroll (Closes #9393) 2016-05-04 23:03:44 +06:00
2a48e6f01a [yandexmusic:playlist] Respect track order for long (>150) playlists 2016-05-04 22:45:01 +06:00
203a3c0e6a [yandexmusic:playlist] Make title optional 2016-05-04 22:35:28 +06:00
d36724cca4 [yandexmusic:playlist] Remove unused imports 2016-05-04 22:34:37 +06:00
15fc0658f7 [yandexmusic:playlist] Modernize 2016-05-04 22:33:29 +06:00
e960c3c223 [yandexmusic:playlist] Improve extraction (Closes #6801) 2016-05-04 22:25:39 +06:00
bc7e77a04b [vevo] Use raise_geo_restricted 2016-05-03 23:18:36 +06:00
964f49336f [aol] Improve _VALID_URL (Closes #9381) 2016-05-03 21:24:51 +06:00
57d8e32a3e [xfileshare] Add support for streamin.to 2016-05-03 16:58:11 +06:00
4174552391 [xfileshare] Refactor _VALID_URL and remove ded sites 2016-05-03 15:35:32 +06:00
80bc4106af [xfileshare] Add support for thevideobee.to (Closes #9374) 2016-05-03 15:09:23 +06:00
7759be38da [xiami] Detect georestriction and skip tests 2016-05-03 16:19:43 +08:00
a0a309b973 [kuwo:category] Fix description and update test 2016-05-03 16:06:28 +08:00
c587cbb793 improved performance by extracting accented chars to top level 2016-05-03 10:40:30 +10:00
6c52a86f54 [README.md] Update creator description 2016-05-02 21:32:57 +06:00
8a92e51c60 [extractor/common] Relax wording for creator metafield 2016-05-02 21:31:35 +06:00
f0e14fdd43 [YoutubeDL] Skip non-relevant field types when building output template 2016-05-02 20:05:06 +06:00
df5f4e8888 [vevo] Remove superfluous code 2016-05-02 18:47:35 +06:00
7960b0563b [YoutubeDL] Properly process unable-to-download-error on python2 2016-05-02 18:35:50 +06:00
5c9ced9504 [vevo] Improve genre extraction 2016-05-02 18:19:00 +06:00
31c4448f6e Instead of replacing accented characters with an underscore when sanitizing file names in restricted mode, replace them with their non-accented equivalents fixes #9347 2016-05-02 13:25:12 +10:00
79a2e94e79 Instead of replacing accented characters with an underscore when sanitizing file names in restricted mode, replace them with their non-accented equivalents fixes #9347 2016-05-02 13:21:39 +10:00
686cc89634 [discovery] Fix typo 2016-05-02 07:07:35 +06:00
9508738f9a [vevo] Extract featured artist 2016-05-02 03:36:40 +06:00
78a3ff33ab [vevo:playlist] Add fallback for playlist id 2016-05-02 03:29:48 +06:00
881dbc86c4 [vevo] Extract track related metafields and add artists to title (Closes #1684) 2016-05-02 03:28:58 +06:00
8e7d004888 [vevo] Add test for video only available via webpage 2016-05-02 03:06:48 +06:00
9618c44824 [vevo] Extract video versions from webpage as a last resort (Closes #8426, closes #9366) 2016-05-02 02:58:20 +06:00
516ea41a7d [vevo] Fix _call_api 2016-05-02 02:54:50 +06:00
e2bd301ce7 [vevo:playlist] Fix genre playlists 2016-05-02 01:00:42 +06:00
0c9d288ba0 [vevo:playlist] Remove debug params 2016-05-02 00:50:31 +06:00
e0da32df6e [vevo:playlist] Add extractor (Closes #9334, closes #9364) 2016-05-02 00:48:26 +06:00
174aba3223 release 2016.05.01 2016-05-01 10:19:14 +02:00
0d66bd0eab [downloader/hls] Delegate extraction to ffmpeg when unsupported features detected 2016-05-01 13:56:51 +06:00
4bd143a3a0 [postprocessor/ffmpeg] Simplify metadata preparation and add track related metafields (Closes #9357) 2016-05-01 10:56:54 +06:00
6f27bf1c74 Credit @blahgeek for xiami (#9079) 2016-05-01 08:08:51 +06:00
68bb2fef95 [tagesschau] Restrict playlist entry regex 2016-05-01 07:15:23 +06:00
854cc54bc1 [tagesschau] Expand video id 2016-05-01 07:01:55 +06:00
651ad35ce0 [tagesschau] Relax _VALID_URL 2016-05-01 06:57:19 +06:00
6a0f9a24d0 [tagesschau] Separate player extractor 2016-05-01 06:45:44 +06:00
9cf79e8f4b [ccc] improve extraction 2016-05-01 01:45:17 +01:00
2844b09336 [tagesschau] Fix article media ids 2016-05-01 04:42:05 +06:00
1a2b377cc2 [tagesschau] Fix audio support 2016-05-01 04:38:46 +06:00
4c1b2e5c0e [tagesschau] Add support for playlists 2016-05-01 04:18:56 +06:00
9e1b96ae40 [rtlnl] Match formats only by height 2016-05-01 03:20:36 +06:00
fc35cd9e0c [tagesschau] Relax _VALID_URL 2016-05-01 02:56:32 +06:00
339fe7228a [tagesschau] Update _FORMATS map 2016-05-01 02:56:32 +06:00
ea7e7fecbd [discovery] remove unused imports 2016-04-30 21:55:28 +01:00
d00b93d58c [discovery] extract more info using BrightcoveNewIE 2016-04-30 21:49:32 +01:00
93f7a31bf3 [discovery] extract subtitle 2016-04-30 20:51:32 +01:00
33a1ec950c [discovery] extract http formats 2016-04-30 20:51:32 +01:00
4e0c0c1508 [xiami] Improve extraction (Closes #9079)
* Switch to JSON source
* Add abstract IE for playlists
* Extract more track related metadata
2016-04-30 21:50:23 +06:00
89c0dc9a5f [xiami] Add xiami extractor 2016-04-30 21:48:40 +06:00
f628d800fb [ted] add support for youtube embeds and update tests 2016-04-30 16:34:57 +01:00
11fa3d7f99 [ted] extract all http formats 2016-04-30 15:44:30 +01:00
d41ee7b774 [vlive] Pass Referer as bytestring (Closes #9352) 2016-04-30 19:22:42 +06:00
e0e9bbb0e9 [pbs] extract srt and vtt subtitles 2016-04-30 14:02:17 +01:00
7691184a31 [pbs] remove duplicate format 2016-04-30 12:57:30 +01:00
35cd2f4c25 [pbs] extract only the formats that we know that they will be available as http format
https://projects.pbs.org/confluence/display/coveapi/COVE+Video+Specifications
2016-04-30 11:32:13 +01:00
350d7963db [pbs] fix the least bitrate http url construction 2016-04-30 11:12:11 +01:00
cbc032c8b7 [pbs] extract all http formats 2016-04-30 01:24:36 +01:00
69c4cde4ba [wsj] improve extraction 2016-04-29 21:37:05 +01:00
ca278a182b [rtlnl] Replace test 2016-04-30 02:07:29 +06:00
373e1230e4 [rtlnl] Clarify tests 2016-04-30 01:50:26 +06:00
cd63d091ce [rtlnl] Fix tests 2016-04-30 01:48:14 +06:00
0571ffda7d [rtlnl] Improve extraction (Closes #9329)
* Make hls extraction non fatal and revert ext
* Extract progressive formats' metadata from corresponding hls formats
2016-04-30 01:43:39 +06:00
5556047465 [rtlnl] Update 720p PG_URL_TEMPLATE
- Fixed the format_id for the 720p progressive videostream and added the video's resolution.
- The adaptive videostreams have the m3u8-extension, so I removed the confusing mp4-extension in order to make a better distinction between the these and the progressive videostreams.
2016-04-30 01:43:13 +06:00
65a3bfb379 [dfb] extract m3u8 formats 2016-04-29 19:21:17 +01:00
cef3f3011f [funimation] Detect blocking and support CloudFlare cookies 2016-04-30 00:17:09 +08:00
e9c6cdf4a1 [common] Fix format_id construction for HLS 2016-04-29 22:50:16 +08:00
00a17a9e12 [crunchyroll] Sort formats 2016-04-29 19:44:10 +06:00
8312b1a3d1 [crunchyroll] Add even more relaxed fmt fallback 2016-04-29 19:43:53 +06:00
6ff4469528 [crunchyroll] Relax fmt regex 2016-04-29 19:39:27 +06:00
68835d687a Merge branch 'Kagami-vlive-hls' 2016-04-29 19:30:51 +08:00
9d186afac8 [vlive] Coding style and PEP8 2016-04-29 19:29:50 +08:00
151d98130b Merge branch 'vlive-hls' of https://github.com/Kagami/youtube-dl into Kagami-vlive-hls 2016-04-29 19:26:39 +08:00
b24d6336a7 [vlive] Add support for live videos 2016-04-29 14:22:50 +03:00
065216d94f [crunchyroll] reduce requests for formats extraction 2016-04-29 11:46:42 +01:00
67167920db [viewlift] replace SnagFilms extractors
- add support for other sites that use the same logic
- improve format extraction and sorting
2016-04-29 11:24:10 +01:00
14638e2915 [sexykarma] Rename to WatchIndianPornIE and fix extraction 2016-04-29 18:17:08 +08:00
1910077ed7 Revert "[sexykarma] Remove the extractor"
This reverts commit 31ff3c074e.
2016-04-29 17:59:23 +08:00
5819edef03 [ooyala] Skip an invalid test
Ooyala is used by lots of extractors and its correctness can be verified
by these websites.
2016-04-29 14:27:15 +08:00
f5535ed0e3 [orf] Skip the expired test 2016-04-29 14:24:07 +08:00
31ff3c074e [sexykarma] Remove the extractor
Its domain name is on sale.

Closes #9317
2016-04-29 13:36:52 +08:00
72670c39de [arte:+7] Fix typo in _VALID_URL 2016-04-29 04:46:23 +06:00
683d892bf9 [viewster] Remove unused import 2016-04-29 01:30:53 +06:00
497971cd4a [yandexmusic] Clarify blockage even more 2016-04-29 01:28:07 +06:00
e757fb3d05 [crunchyroll] improve extraction
- extract more metadata(series, episode, episode_number)
- reduce duplicate requests for extracting formats
- remove duplicate formats
2016-04-28 18:42:20 +01:00
0ba9e3ca22 [viewster] extract formats for videos with multiple audios/subtitles 2016-04-28 17:45:09 +01:00
4b53762914 [yandexmusic] Clarify blockage 2016-04-28 21:45:33 +06:00
eebe6b382e [yandexmusic] Improve error handling 2016-04-28 21:37:34 +06:00
0cbcbdd89d [nuvid] Fix extraction
Closes #7620
2016-04-28 17:51:20 +08:00
7f776fa4b5 [yandexmusic] Skip tests as Travis CI blocked 2016-04-28 17:08:41 +08:00
eb5ad31ce1 Merge branch 'pmrowla-mwave-meetgreet' 2016-04-28 16:03:43 +08:00
a5941305b6 [mwave] Coding style 2016-04-28 16:03:08 +08:00
f8dddaf456 Merge branch 'mwave-meetgreet' of https://github.com/pmrowla/youtube-dl into pmrowla-mwave-meetgreet 2016-04-28 15:56:32 +08:00
618c71dc64 [cloudy] New domain name for the test_cloudy_1
I'm sure whether videoraj.ch still works or not, so keep it.
2016-04-28 15:46:00 +08:00
52af8f222b [cwtv] Relax _VALID_URL (Closes #9327) 2016-04-28 04:01:21 +06:00
3cc8649c9d [20min] Detect embedded YouTube videos
Fixes #9331
2016-04-28 02:58:11 +08:00
dcf094d626 [theplatform] Fix for Python 3.2
test_AENetworks{,_1} fails as in Python < 3.3, binascii.a2b_* functions
accepts only bytes-like objects
2016-04-27 18:35:33 +08:00
5b5d7cc11e [mwave] Add Mwave Meet & Greet extractor 2016-04-27 15:57:17 +09:00
2ac2cbc0a3 [malemotion] Remove the extractor
Announcement from their homepage:

```
MaleMotion is closed

After another system crash, I'm forced to close the site

This week all content will be erased

Don't forget to cancel your subscription if any !
```

Closes #9311.
2016-04-27 13:55:32 +08:00
a7e03861e8 [scivee] Skip the test
Not accessible from either Travis CI or my machine.

Closes #9315
2016-04-27 13:52:04 +08:00
046ea04a7d [README.md] Mention mpv 2016-04-27 00:22:08 +06:00
7464360379 [README.md] Add FAQ entry on output template conflicts 2016-04-27 00:16:48 +06:00
175c2e9ec3 [youtube:search_url] Reimplement in terms of youtube:playlistbase 2016-04-26 22:29:29 +06:00
f1f879098a [viewster] extract more metadata for http formats 2016-04-26 13:40:40 +01:00
c9fd530670 [ok] Extract start time 2016-04-25 22:15:15 +06:00
749b0046a8 [ok] Allow embeds without title (Closes #9303) 2016-04-25 22:05:47 +06:00
e3de3d6f2f [normalboots] Fix extraction
Now it's using ScreenwaveMedia
2016-04-25 23:49:12 +08:00
ad58942d57 [muzu] Remove extractor
MUZU is shutting down in October 2015. [1]

[1] http://www.musicbusinessworldwide.com/youtube-rival-muzu-is-heading-into-liquidation/
2016-04-25 23:35:05 +08:00
4645432d7a [eagleplatform] Checking direct HTTP links
Sometimes they fail with 404
2016-04-25 22:48:17 +08:00
6bdc2d5358 [mitele] Comment out unstable MD5
Also Akamai f4f fragments
2016-04-25 22:27:25 +08:00
2beff95da5 [nrk] Comment out unstable MD5 checksums
Both are Akamai f4f fragments.
2016-04-25 22:26:19 +08:00
abc1723edd [unistra] Sort formats
Originally URLs are passed to set() and not sorted, so the result is not
deterministic, causing occasional FAILs on Travis CI.
2016-04-25 22:24:40 +08:00
b248e6485b Merge branch 'remitamine-akamai_pv' 2016-04-25 21:02:30 +08:00
d6712378e7 Merge branch 'akamai_pv' of https://github.com/remitamine/youtube-dl into remitamine-akamai_pv 2016-04-25 21:02:02 +08:00
fb72ec58ae [extractor/common] do not process f4m manifest that contain akamai playerVerificationChallenge 2016-04-25 13:37:03 +01:00
c83a352227 [openload] Make thumbnail optional 2016-04-25 00:26:06 +06:00
e9063b5de9 [openload] Add test 2016-04-25 00:22:55 +06:00
594b0c4c69 [openload] Fix ext extraction 2016-04-25 00:03:29 +06:00
eb9ee19422 [utils] Allow None mimetypes in mimetype2ext 2016-04-25 00:03:12 +06:00
a1394b820d [openload] Fix title extraction (Closes #9298) 2016-04-25 00:01:37 +06:00
aa9dc24f5a [douyutv] Improve extraction and update tests
The JSON API sometimes return HTML pages with errors
2016-04-24 23:52:17 +08:00
51762e1a31 [xminus] Fix extraction (closes #9228) 2016-04-24 23:21:45 +08:00
8b38f2ac40 release 2016.04.24 2016-04-24 17:06:46 +02:00
a82398bd72 [kwuo:song] Fix extraction and update the test 2016-04-24 22:20:45 +08:00
c14dc00df3 [viewster] improve http formats extraction 2016-04-24 14:34:28 +01:00
03dd60ca41 [kuwo:category] Fix the test
Sometimes there are 24 songs and sometimes 30 lol
2016-04-24 21:16:06 +08:00
0738187f9b [ThePlatform] Fix tests failed since 79ba9140dc 2016-04-24 20:46:06 +08:00
a956cb6306 [onionstudios] Fix description extraction
\1 does not work in []. Fixes test_Generic_75
(http://www.clickhole.com/video/dont-understand-bitcoin-man-will-mumble-explanatio-2537)
2016-04-24 20:41:17 +08:00
a8062eabcd [mwave] Skip checking unstable MD5
On my PC the checksum is 02eda6d09fb63131a17a8d44e6237463, while a
recent Travis CI build
(https://travis-ci.org/rg3/youtube-dl/jobs/125341081) shows it's
c930e27b7720aaa3c9d0018dfc8ff6cc
2016-04-24 20:05:24 +08:00
2a7dee8cc5 [yahoo] Improve error detection and update tests 2016-04-24 18:12:16 +08:00
d9ed362116 [yahoo] Extract all <iframe>s
Fixes test_yahoo_6

(https://ca.finance.yahoo.com/news/hackers-sony-more-trouble-well-154609075.html)
2016-04-24 17:46:25 +08:00
4f54958097 [yahoo] Update some tests
One has new fields as ThePlatformIE changed, and others have changed
files.
2016-04-24 17:29:01 +08:00
2a7c38831c [yahoo] Extend _VALID_URL and fix extraction
Closes #9271
2016-04-24 17:01:18 +08:00
949b6497cc [generic] Unescape the video URL
Fixes #9279
2016-04-24 16:25:37 +08:00
2c21152ca7 [README.md] Document track metafields in output template 2016-04-24 12:22:18 +06:00
fda9a1ca9e [viewster] simplify qualities_basename regex 2016-04-24 03:06:46 +01:00
864d5e7231 [viewster] extract all http formats 2016-04-24 02:32:56 +01:00
ccff2c404d [ffmpeg] Fix embedding subtitles (#9063)
Changed command line parameters for ffmpeg when embedding subtitles.
Changed to ‘-map 0:v -c:v copy -map 0:a -c:a copy’
2016-04-24 00:08:02 +08:00
5448b781f6 [dplay] Sign unsigned final download hls URLs 2016-04-23 17:28:45 +06:00
e239413fbc [dplay] Extract subtitles (Closes #9284) 2016-04-23 16:50:31 +06:00
fd0ff8bad8 [dplay] Improve extraction and document workarounds and tests 2016-04-23 16:36:17 +06:00
397ec446f3 [dplay] Try secure api for no tld (Closes #9282) 2016-04-23 15:59:30 +06:00
14f7a2b8af [WDRMaus] switch current show to new WDR extractor (fixes #8562)
It seems that the "current show" already uses the new WDR video-player,
while all the others videos still use the old player.

I just added the current show URL to the normal WDR-extractor, which
works fine. This commit needs my changes from PR #8842 that fix the
support for WDR.
2016-04-23 11:53:22 +02:00
c0837a12c8 [WDR] complete overhaul after relaunch of the site
The WDR relaunched their site on 2016-02-23 which not only changed the
URL-schema completely but also the layout of their pages.

Apparently the whole "mediathek" now runs on the wdr-domain, so no
separate URL for funkhauseuropa anymore.
There seems to be no explicit handling of video-sizes on the page or in
the URLs anymore. There seems to be only one size for HTML5, but still
several sizes for flash. The extractor adds all to the list of formats.

There is no metadata for the HTML5-stream, so that the best flash-stream
will always be considered as the "best" format. At least in my tests
this seemed to be true anyway.
2016-04-23 11:42:18 +02:00
29a7e8f6f8 [nhl] Add new extractor(closes #8419)(closes #8798) 2016-04-22 20:18:27 +01:00
eb01e97e10 [youku] Skip streams with channel_type=tail
Fixes #9275

These video segments look like ads and they don't appear in the web
player.
2016-04-23 02:54:09 +08:00
cb7d4d0efd [nbc] add support for today.com(closes #2909) 2016-04-22 18:08:20 +01:00
c80037918b [iqiyi] Improve error detection (#9276) 2016-04-23 00:06:49 +08:00
237a41108a [eagleplatform] extract all http formats 2016-04-22 14:32:38 +01:00
e962ae15d3 [newstube] extract http formats(closes #9253) 2016-04-22 11:26:43 +01:00
7c36ea7d54 [rtbf] improve extraction(fixes #9267) 2016-04-21 22:52:49 +01:00
9260cf1d97 [tubitv] fix extraction(closes #8741) 2016-04-21 20:30:19 +01:00
bdbb8530c7 [vimeo] Pass Referer for check-password request 2016-04-22 00:02:39 +06:00
09a9fadb84 [dump] Remove extractor 2016-04-21 23:31:34 +06:00
bf09af3acb Add --hls-prefer-ffmpeg 2016-04-21 23:02:17 +06:00
88296ac326 [planetaplay] Remove remainings of extractor 2016-04-21 22:57:38 +06:00
870d525848 [options] Remove experimental mark for --hls-prefer-native 2016-04-21 22:44:01 +06:00
6577112890 [planetaplay] Remove extractor (Closes #9256) 2016-04-21 22:33:54 +06:00
1988647dda [tvigle] Skip hls completely (#9259) 2016-04-21 22:15:20 +06:00
a292cba256 [mgtv] Fix _VALID_URL and add localized name 2016-04-22 00:07:43 +08:00
982e518a96 [dispeak] Rename DigitalSpeaking to DigitallySpeaking 2016-04-22 00:07:43 +08:00
748e730099 [dispeak] Several fixes 2016-04-22 00:07:43 +08:00
b6c0d4f431 Merge pull request #9110 from remitamine/parse_duration
[utils] imporove parse_duration to handle more formats
2016-04-21 22:53:16 +07:00
acaff49575 [utils] imporove parse_duration to handle more formats 2016-04-21 16:34:54 +01:00
1da19488f9 [mgtv] Add new extractor (closes #9212) 2016-04-21 23:29:51 +08:00
442c4d361f [dispeak/gdcvault] Add the test case from #5784 2016-04-21 19:47:10 +08:00
ec59d657e7 [dispeak] Add new extractor
Both GDCVault and GPUTechConf uses the service of DigitalSpeaking.
2016-04-21 19:36:33 +08:00
99ef96f84c [gdcvault] Fix for videos with hard-coded hostnames
Fixes #9248
2016-04-21 18:07:03 +08:00
4dccea8ad0 [streetvoice] Fix extraction
The old API results in URLs with HTTP 403 from time to time.

Hopefully fixes #9219.
2016-04-21 13:07:53 +08:00
2c0d9c6217 [extractor/common] Allow empty post data 2016-04-21 13:06:06 +08:00
12a5134596 [tvigle] Fix extraction (Closes #9259) 2016-04-20 23:52:41 +06:00
16e633a5d7 [quickvid] Remove extractor (Closes #9258) 2016-04-20 23:29:02 +06:00
494ab6db73 [youtube] Capture and output login error message 2016-04-20 22:14:32 +06:00
107701fcfc [people] Remove bogus comment 2016-04-20 03:40:02 +06:00
f77970765a [people] Add extractor 2016-04-20 03:37:23 +06:00
81215d5652 release 2016.04.19 2016-04-19 03:03:52 +02:00
241a318f27 [vimeo] Improve _VALID_URL (Closes #9229) 2016-04-18 21:40:28 +06:00
4fdf082375 [theonion] Remove extractor (Closes #9220)
It now uses generic onionstudios embed
2016-04-17 23:12:23 +06:00
1b6182d8f7 [youtube:playlist] Fetch all the videos in a mix (fixes #3837)
Since there doesn't seem to be any indication, it stops when there aren't new videos in the webpage.
2016-04-17 17:07:57 +02:00
7bab22a402 [vice] remove unused import and variable 2016-04-17 14:06:19 +01:00
0f97fb4d00 [musicplayon] Relax _VALID_URL and improve metadata extraction
In r'pl=\d+&play=\d+' pages, several metadata items are missing

Closes #9222.
2016-04-17 17:24:33 +08:00
b1cf58f48f [musicplayon] Fix extraction (closes #9222) 2016-04-17 15:08:51 +08:00
3014b0ae83 Merge pull request #9195 from remitamine/ffmpeg-pipe
[downloader/external] enable piping for FFmpegFD(closes #2124)
2016-04-16 22:00:49 +01:00
b9f2fdd37f [ffmpeg] Clarify rationale for pipe(-) exclusion in _ffmpeg_filename_argument 2016-04-16 21:50:13 +01:00
bbb3f730bb [onionstudios] extract m3u8 formats 2016-04-16 20:53:13 +01:00
d868f43c58 [ffmpeg] check for - file name in _ffmpeg_filename_argument 2016-04-16 19:45:56 +01:00
21525bb8ca [kuwo:category] Update the test
Now the webpage says there are 24 songs.
2016-04-17 02:38:05 +08:00
d8f103159f [nerdist] Remove extractor
It now uses brightcove
2016-04-17 00:16:31 +06:00
663ee5f0a9 [vice] extract youtube embed 2016-04-16 17:49:39 +01:00
b6b950bf58 [cbs] Remove unused import 2016-04-16 22:47:10 +06:00
11e60fcad8 [extractor/generic] Improve instagram embeds (Closes #9213) 2016-04-16 22:39:20 +06:00
c23533a100 [instagram] Add support for iframe embeds 2016-04-16 22:31:05 +06:00
0dafea02e6 [instagram] Add support for embed URLs 2016-04-16 22:23:08 +06:00
5d6360c3b7 [mooshare] Remove extractor 2016-04-16 21:31:50 +06:00
5e5c30c3fd [mdr] Fix extraction and update tests
It's strange that the date is changed. Anyway, new data matches what the
webpage says.
2016-04-16 21:57:28 +08:00
9154c87fc4 [huffpost] Fix a typo 2016-04-16 21:41:22 +08:00
ef0e4e7bc0 [generic] Fix test_Generic_2
Now a HEAD request returns 400 Bad Request
2016-04-16 19:44:45 +08:00
67d46a3f90 [ustream] Fix /embed/ URLs and add a test 2016-04-16 19:39:25 +08:00
bec47a0748 [tudou] Improve error detection (closes #9175) 2016-04-16 19:11:25 +08:00
36b7d9dbfa [twitter] Don't check /cards/ URLs
Fixes #9181

In this tweet, there are two cards:
1. https://twitter.com/i/cards/tfw/v1/719944006306701313
   This shows #TeamCap vs. #TeamIronMan
2. https://twitter.com/i/videos/tweet/719944021058060289
   This is the real video and can be handled by TwitterCardIE

In all current test_Twitter* tests, /videos/tweet/ approach works fine.
2016-04-16 18:57:50 +08:00
8c65e4a527 [bbc] Fix a test 2016-04-16 18:00:19 +08:00
6ad2ef8b7c [audiomack] Update the test
The original test raises 404
2016-04-16 17:54:39 +08:00
00b426d66d [varzesh3] Add md5 to the test 2016-04-16 17:41:56 +08:00
0de968b584 [newgrounds] Support videos (closes #9138) 2016-04-16 17:41:56 +08:00
0841d5013c [cbs] do not catch Exceptions raised by by _extract_theplatform_smil 2016-04-16 10:25:59 +01:00
a71fca8577 [theplatform] remove _sort_formats from _extract_theplatform_smil 2016-04-16 10:23:56 +01:00
ee94e7e66d [varzesh3] Fix metadata extraction (closes #9197) 2016-04-16 17:13:22 +08:00
759e37c9e6 [gazeta] Relax _VALID_URL and update tests
Closes #9196
2016-04-16 16:48:47 +08:00
ae65567102 [eagleplatform] Fix error handling 2016-04-16 16:47:16 +08:00
c394b4f4cb [puls4] Fix error detection (#9194) 2016-04-16 16:22:44 +08:00
260c7036ba [sportbox] Fix SportBoxEmbedIE
Also fixes test_Generic_29 (http://www.vestifinance.ru/articles/25753)
2016-04-16 16:13:14 +08:00
f74197a074 [cbs] extract rtmp formats 2016-04-15 22:38:37 +01:00
f3a58d46bf [youtube:user] check if the url didn't match only the other youtube extractors 2016-04-15 19:06:13 +01:00
b6612c9b11 [karaoketv] Fix extraction 2016-04-15 21:26:54 +06:00
7e176effb2 [iqiyi] Also suuport pps.tv URLs
PPS is acquired by Baidu and merged with iQiyi in 2013 [1]. Now they
have the same page layouts.

[1] http://www.chinanews.com/it/2013/05-07/4792526.shtml
2016-04-15 22:39:18 +08:00
4a252cc2d2 [karaoketv] Update and mark as not _WORKING 2016-04-15 21:49:17 +08:00
f0ec61b525 [huffpost] Fix extraction 2016-04-15 20:55:56 +08:00
66d40ae3a5 Merge pull request #9041 from kasper93/master
[generic] Add support for LiveLeak embeds
2016-04-15 17:23:55 +08:00
e6da9240d4 [mixcloud:stream] Add new extractor
Closes #7633
2016-04-15 17:14:17 +08:00
dd91dfcd67 [mixcloud] Fix extraction by decrypting play info
Fixes #7521
2016-04-15 15:48:22 +08:00
c773082692 Merge branch 'Phaeilo-mixcloud' 2016-04-15 14:33:04 +08:00
9c250931f5 [mixcloud] Improve and simplify mixcloud:user and mixcloud:playlist 2016-04-15 14:32:02 +08:00
56f1750049 [tdslifeway] Use the new Brightcove API
Thanks for @remitamine's suggestion.
2016-04-15 04:28:54 +08:00
f2159c9815 [wayofthemaster] Remove extractor
Now it's using YouTube embeds.
2016-04-15 04:02:23 +08:00
b0cf2e7c1b [ubu] Remove extractor
1. Videos on ubu.com are now hosted on Vimeo
2. The duration is far from correct, and may not exist on other videos
   (For example http://ubu.com/film/hammons_king.html)
2016-04-15 03:48:23 +08:00
74b47d00c3 [xboxclips] Use http:// URL
xboxclips has misconfigured certificates
2016-04-15 03:30:38 +08:00
8cb57bab8e [ministrygrid] Fix extraction and modernize 2016-04-15 02:48:12 +08:00
e1bf277e19 [tdslifeway] Add TDSLifewayIE
Used by MinistryGridIE
2016-04-15 02:48:12 +08:00
ce599d5a7e [downloader/external] enable piping for FFmpegFD(closes #2124) 2016-04-14 18:49:02 +01:00
9e28538726 [arte:creative] Improve _VALID_URL 2016-04-14 21:54:41 +06:00
404284132c [arte:info] Add extractor (Closes #9182) 2016-04-14 21:52:05 +06:00
5565be9dd9 [aol] relex _VALID_URL regex 2016-04-14 08:47:55 +01:00
b3a9474ad1 Merge branch 'mixcloud' of https://github.com/Phaeilo/youtube-dl into Phaeilo-mixcloud 2016-04-14 15:31:58 +08:00
86475d59b1 [metacritic] Add a new valid test case 2016-04-14 15:12:59 +08:00
73d93f948e [lecture2go] Fix extraction
RTSP stream fails to download. Seems it's a mpv bug as direct playback
works well:

$ mpv --ytdl-format rtsp https://lecture2go.uni-hamburg.de/veranstaltungen/-/v/17473
2016-04-14 15:08:01 +08:00
f5d8743e0a [downloader/rtsp] Print the command 2016-04-14 15:07:31 +08:00
d1c4e4ba15 [laola1tv] Improve error detection and skip an invalid test 2016-04-14 14:11:28 +08:00
f141fefab7 [karrierevideos] Fix extraction
The server serves malformed header "Content Type: text/xml" for the XML
request (it should be Content-Type but not Content Type). Python 3.x,
which uses email.feedparser rejects such headers. As a result,
Content-Encoding header is not parsed, so the returned content is kept
not decompressed, and thus XML parsing error.
2016-04-14 14:06:05 +08:00
8334637f4a Make tbr field 'int' rather than 'tuple'
Closes #9180.
2016-04-13 14:29:34 +02:00
b0ba11cc64 release 2016.04.13 2016-04-13 08:02:03 +02:00
b8f67449ec [generic] Add support for LiveLeak embeds 2016-04-13 01:54:19 +02:00
75af5d59ae [netease] Skip all tests: completely georestricted 2016-04-13 04:52:07 +08:00
b969d12490 Credit @Phaeilo for presstv (#7113) 2016-04-13 01:52:50 +06:00
6d67169509 [mixcloud] improved extraction of user description 2016-04-12 21:18:13 +02:00
dcaf00fb3e [mixcloud] support older urllib versions 2016-04-12 21:18:13 +02:00
f896e1ccef [mixcloud] fixed some tests 2016-04-12 21:18:13 +02:00
c96eca426b [mixcloud] Added support for user uploads, playlists, favorites and listens.
Fixes #3750 and #5272
2016-04-12 21:18:13 +02:00
466a614537 [youtube:playlist] Recognize popular uploads playlist as mix (Closes #9170) 2016-04-12 21:38:31 +06:00
ffa2cecf72 [ard] Change subtitles extension to ttml (Closes #9169)
ttml is now served instead of srt
2016-04-12 21:20:31 +06:00
a837416025 [jadorecettepub] Remove extractor: website gone 2016-04-12 18:30:53 +08:00
c9d448876f [izlesene] Fix extraction
description may be absent
2016-04-12 18:29:28 +08:00
8865b8abfd [howstuffworks] Skip a broken test case 2016-04-12 17:30:14 +08:00
c77a0c01cb [groupon] Fix extraction 2016-04-12 17:26:09 +08:00
12355ac473 [goshgay] Fix extraction
isFamilyFriendly no longer exists in the webpage and I can't find
another indicator.
2016-04-12 17:23:00 +08:00
49f523ca50 [mixcloud] Capture error message (#9156) 2016-04-11 20:45:58 +06:00
4a903b93a9 Revert "[openclassroom] Add new extractor(closes #9147)"
This reverts commit 13267a2be3.
2016-04-11 14:44:35 +01:00
13267a2be3 [openclassroom] Add new extractor(closes #9147) 2016-04-11 14:24:08 +01:00
134c207e3f [arte.tv:embed] Extended support (#2620) 2016-04-11 19:32:27 +08:00
0f56bd2178 Merge branch 'Phaeilo-presstv' 2016-04-11 16:17:05 +08:00
dfbc7f7f3f [presstv] Improve and simplify 2016-04-11 16:14:07 +08:00
7d58ea7c5b Merge branch 'presstv' of https://github.com/Phaeilo/youtube-dl into Phaeilo-presstv 2016-04-11 15:48:10 +08:00
452908b257 [telebruxelles] Fix extraction (Closes #9142) 2016-04-11 00:06:05 +06:00
5899e988d5 [glide] Improve extraction and extract upload info 2016-04-10 23:56:23 +06:00
4a121d29bb [glide] Fix extraction (Closes #9141) 2016-04-10 23:45:17 +06:00
7ebc36900d [jwplatform:base] Improve subtitles extraction 2016-04-10 22:55:07 +06:00
d7eb052fa2 [screencastomatic] Add duration to test 2016-04-10 22:48:04 +06:00
a6d6722c8f [jwplatform:base] Extract duration 2016-04-10 22:47:38 +06:00
66fa495868 [screencastomatic] Fix extraction (Closes #9136) 2016-04-10 22:37:14 +06:00
443285aabe [ebaumsworlds] Update _VALID_URL (Closes #9135) 2016-04-10 22:15:11 +06:00
de728757ad [presstv] Refactored extractor. 2016-04-10 16:36:44 +02:00
f44c276842 [extractor/extractors] Remove non-existant imports 2016-04-10 19:21:58 +06:00
a1fa60a934 [cliprs] Add extractor (Closes #9099) 2016-04-10 18:43:40 +06:00
49caf3307f [extractor/common] Remove irrelevant comment 2016-04-10 17:10:27 +06:00
6a801f4470 [test/InfoExtractors] add test for _download_json 2016-04-09 23:18:41 +02:00
61dd350a04 [1tv] Fix extraction (Closes #9103) 2016-04-10 03:02:35 +06:00
eb9c3edd5e [test/utils] Add test for date_from_str 2016-04-09 22:40:05 +02:00
95153a960d [presstv] updated extractor and tests to work with current PressTV website 2016-04-09 16:14:05 +02:00
6c4c7539f2 [test/helper] Check got values to be strings for md5: fields
Seen in PBSIE tests
2016-04-09 22:04:48 +08:00
c991106706 [videodetective] Adapt to InternetVideoArchiveIE 2016-04-09 21:47:35 +08:00
dae2a058de [rottentomatoes] Adapt to InternetVideoArchiveIE 2016-04-09 21:47:12 +08:00
c05025fdd7 [internetvideoarchive] Fix extraction and support json URLs 2016-04-09 21:46:51 +08:00
bfe96d7bea [presstv] Added extractor PressTV.
Fixes #7060
2016-04-09 14:55:54 +02:00
ab481b48e5 [funnyordie] Relax M3U8 URL matching
Also, m3u8_url extraction should be fatal as all formats depends
directly or indirectly on it.

This change fixes test_Generic_26 and TestFunnyOrDieSubtitles
2016-04-09 20:17:35 +08:00
92c7f3157a [aol] Add coding cookie 2016-04-09 17:32:23 +06:00
cacd996662 [utils] Don't touch URLs if not necessary
Fix test_Generic_15 (Google redirect)
2016-04-09 19:27:54 +08:00
bffb245a48 [aol] add support for videos with vidible IDs(closes #9124) 2016-04-09 10:51:23 +01:00
680efb6723 Merge pull request #8497 from jaimeMF/lazy-load
Add experimenta lazy loading of info extractors
2016-04-09 14:08:13 +08:00
5a9858bfa9 setup.py: add command for building the lazy_extractors module 2016-04-08 21:50:54 +02:00
8a5dc1c1e1 lazy extractors: Initialize the real info extractor
According to the docs '__init__' is only called automatically if '__new__' returns an instance of the original class.
2016-04-08 21:50:54 +02:00
e0986e31cf lazy extractors: Output if it's enabled in the verbose log 2016-04-08 21:50:54 +02:00
6b97ca96fc lazy extractors: Style fixes
* Sort extractors alphabetically
* Add newlines when needed (youtube_dl/extractors/lazy_extractors.py pass the flake8 test now)
2016-04-08 21:50:54 +02:00
c1ce6acdd7 lazy extractors: Fix building with python2.6 2016-04-08 21:50:07 +02:00
0d778b1db9 lazy extractors: specify the encoding
When building with python3 the unicode characters are not escaped, python2 needs to know the encoding.
2016-04-08 21:50:07 +02:00
779822d945 Add experimental support for lazy loading the info extractors
'make lazy-extractors' creates the youtube_dl/extractor/lazy_extractors.py (imported by youtube_dl/extractor/__init__.py), which contains simplified classes that only have the 'suitable' class method and that load the appropiate class with the '__new__' method when a instance is created.
2016-04-08 21:50:07 +02:00
1b3d5e05a8 Move the extreactors import to youtube_dl/extractor/extractors.py 2016-04-08 21:47:51 +02:00
e52d7f85f2 Delay initialization of InfoExtractors until they are needed 2016-04-08 21:43:24 +02:00
568d2f78d6 [tnaflix] Fix metadata extraction 2016-04-09 00:27:24 +06:00
2f2fcf1a33 [tnaflix] Fix extraction (Closes #9074) 2016-04-08 23:34:59 +06:00
bacec0397f [extractor/common] Relax _hidden_inputs 2016-04-08 23:33:45 +06:00
3c6c7e7d7e [gdcvault] Fix extraction (Closes #9107, closes #9114) 2016-04-08 23:16:02 +06:00
fb38aa8b53 [extractor/common] Support arbitrary format strings for template based identifiers in mpd manifests (Closes #9119, closes #9120) 2016-04-08 22:48:08 +06:00
18da24634c [democracynow] Improve extraction 2016-04-08 22:27:27 +06:00
a134426d61 [democracynow] Fix tests 2016-04-08 22:21:14 +06:00
a64c0c9b06 [democracynow] Make description optional (Closes #9115) 2016-04-08 22:15:36 +06:00
56019444cb [novamov] Improve _VALID_URL template (Closes #9116) 2016-04-08 21:26:42 +06:00
a1ff3cd5f9 [acast] fix channel extraction(closes #9117) 2016-04-08 15:15:34 +01:00
9a32e80477 [acast] fix extraction(#9117) 2016-04-08 14:51:00 +01:00
536a55dabd [YoutubeDL] Sanitize single thumbnail URL 2016-04-08 00:17:47 +06:00
ed6fb8b804 [vrt] Add support for direct hls playlists and YouTube (Closes #9108) 2016-04-07 23:22:43 +06:00
3afef2e3fc [beeg] Improve extraction 2016-04-07 22:40:35 +06:00
e90d175436 [yandexmusic] Extract music album metafields (Closes #7354) 2016-04-07 02:56:13 +06:00
7a93ab5f3f [extractor/common] Introduce music album metafields 2016-04-07 02:53:53 +06:00
c41cf65d4a release 2016.04.06 2016-04-06 15:13:08 +02:00
ec4a4c6fcc Makefile: remove ISSUE_TEMPLATE.md from the 'all' target (fixes #9088)
It isn't included in the tar file, causing build failures.
Since it's only used for GitHub, I think we don't need to store it in the tar file.
2016-04-06 14:16:05 +02:00
be0c7009fb Makefile: use full path for the ISSUE_TEMPLATE.md file 2016-04-06 14:09:31 +02:00
92d5477d84 [compat] Handle tuples properly in urlencode()
Fixes #9055
2016-04-06 18:29:54 +08:00
8790249c68 [iqiyi] Improve error detection for VIP-only videos
Closes #9071
2016-04-06 16:12:16 +08:00
416930d450 release 2016.04.05 2016-04-05 18:36:24 +02:00
65150b41bb [deezer] Fix extraction (Closes #9086) 2016-04-05 22:27:33 +06:00
e42f413716 [rte] Improve thumbnail extraction (Closes #9085) 2016-04-05 22:23:20 +06:00
40a056d85d [extractor/__init__] Remove novamov extractor and sort novamov based extractors alphabetically 2016-04-05 21:54:09 +06:00
e7d77efb9d [auroravid] Add extractor (Closes #9070) 2016-04-05 21:52:07 +06:00
995cf05c96 [novamov] Make title fatal 2016-04-05 21:40:43 +06:00
5bf28d7864 [utils] dfxp2srt: add additional namespace
Used by the ZDF subtitles (#9081).
2016-04-04 20:46:35 +02:00
8c7d6e8e22 [zdf] Extract subtitles (closes #9081) 2016-04-04 20:44:06 +02:00
6d4fc66bfc [youtube] Add support for zwearz (Closes #9062) 2016-04-04 02:26:20 +06:00
23576edbfc [brightcove:legacy] skip None value for uploader_id 2016-04-02 21:31:21 +01:00
4d4cd35f48 [brightcove:legacy] extract uploader_id as a string 2016-04-02 20:55:44 +01:00
3aac9b2fb1 [nowness] update tests 2016-04-02 18:57:15 +01:00
e47d19e991 [brightcove:new] extract subtitles and strip video title 2016-04-02 18:57:15 +01:00
41f5492fbc [brightcove:legacy] improve format extraction and extract uploader_id, duration and timestamp 2016-04-02 18:57:15 +01:00
2defa7d75a [instagram:user] Fix extraction (fixes #9059)
The URL for the next page was incorrect and we always got the same page, therefore it got trapped in an infinite loop.
2016-04-02 18:03:56 +02:00
bbc26c8a01 [bbc] Set vcodec to none for audio formats 2016-04-02 19:00:38 +06:00
b507cc925b [extractor/common] Carry long line 2016-04-02 18:49:58 +06:00
db8ee7ec05 [extractor/common] Fix numeric identifiers conversion in DASH URL templates 2016-04-02 18:48:05 +06:00
08136dc138 [brightcove] fix format sorting 2016-04-02 10:57:57 +01:00
fe7ef95e91 [cbsinteractive] Add support for ZDNet videos 2016-04-01 23:53:32 +01:00
5f705baf5e [cnet] extract more formats 2016-04-01 20:42:15 +01:00
0750b2491f [ffmpeg] try to convert tt subtitles usng dfxp2srt 2016-04-01 19:47:49 +01:00
df634be2ed [common] prefer using mime type over ext for smil subtitle extraction
the subtitle ext for http://www.cnet.com/videos/download-amazon-prime-movies-and-tv/
is adb_xml while using the mime type it get tt(application/smptett+xml)
2016-04-01 19:47:49 +01:00
6d628fafca [camwithher] Remove extra blank line 2016-04-01 20:45:21 +02:00
0f28777f58 [cbsnews] Remove unused import 2016-04-01 20:43:14 +02:00
329c1eae54 [aenetworks] Make pep8 happy 2016-04-01 20:42:19 +02:00
9aaaf8e8e8 [camwithher] Improve extraction (Closes #8989) 2016-04-01 23:47:27 +06:00
04819db58e [camwithher] Add extractor
Corrected unnecessary test

Sane variable naming

RTMP all .flv & url_id for _download_webpage()

Corrected all outstanding issues, next up is a squash!
2016-04-01 23:44:25 +06:00
79ba9140dc [theplatform] extract timestamp and uploader 2016-04-01 18:07:17 +01:00
75d572e9fb [screencast] Improve title regexes (Closes #9025) 2016-04-01 23:01:55 +06:00
791d6aaecc screencast.com: fallback on page title
When determining the title of the page, use the <title> tag of the page
2016-04-01 23:00:52 +06:00
81de73e5b4 [screencast] Add test 2016-04-01 23:00:45 +06:00
83cedc1cf2 screencast.com: support missing www
The "www." part of the URL is not mandatory
2016-04-01 22:58:16 +06:00
244cd04237 [pluralsight] Remove unnecessary login/password encode 2016-04-01 22:46:46 +06:00
fbdaced256 [lynda] Remove unnecessary login/password encode 2016-04-01 22:45:20 +06:00
a3373823e1 [udemy] Remove unnecessary login/password encode
This is now covered by compat_urllib_parse_urlencode
2016-04-01 22:42:09 +06:00
03caa463e7 [udemy:course] Skip non-video lectures 2016-04-01 22:38:56 +06:00
3f64379eda [movieclips] fix extraction 2016-04-01 16:22:06 +01:00
3e0c3d14d9 [cbs] add base extractor 2016-04-01 10:12:29 +01:00
d8873d4def [aenetworks] improve format extraction 2016-04-01 09:58:02 +01:00
db1c969da5 [theplatform] sign https urls 2016-04-01 09:58:02 +01:00
1e02bc7ba2 release 2016.04.01 2016-04-01 09:07:40 +02:00
63c55e9f22 [cbs] improve extraction(closes #6321) 2016-04-01 07:33:37 +01:00
f9b1529af8 [generic] remove sbnation test(handled by VoxMediaIE) 2016-03-31 23:50:45 +01:00
961fc024d2 [voxmedia] improve sbnation support 2016-03-31 23:33:36 +01:00
b53a06e3b9 [udemy:course] Use new URL format 2016-04-01 02:24:22 +06:00
4ecc1fc638 [howstuffworks] improve extraction 2016-03-31 21:11:58 +01:00
5b012dfce8 [tudou] Improve error handling (closes #8988) 2016-04-01 01:42:16 +08:00
8369942773 [voxmedia] Add new extractor(closes #3182) 2016-03-31 18:36:41 +01:00
86f3b66cec [udemy] Remove unused import 2016-03-31 23:00:11 +06:00
6bb4600717 [udemy:course] Simplify course curriculum downloading 2016-03-31 22:59:19 +06:00
41d06b0424 [extractor/common] Improve _request_webpage
* Do not ignore data, headers and query for Requests
* Default values for headers and query switched to dicts since these are used by urllib itself
2016-03-31 22:58:38 +06:00
15d260ebaa [utils] Use update_Request in http_request 2016-03-31 22:55:49 +06:00
ed0291d153 [utils] Add update_Request 2016-03-31 22:55:01 +06:00
81da8cbc45 [udemy] Switch to api 2.0 (Closes #9035) 2016-03-31 22:05:25 +06:00
5299bc3f91 [beeg] Switch to api v6 (Closes #9036) 2016-03-31 20:42:41 +06:00
c9c39c22c5 [nationalgeographic] add support for channel.nationalgeographic.com urls 2016-03-31 13:47:38 +01:00
d84b48e3f1 [nationalgeographic] improve extraction 2016-03-31 13:44:55 +01:00
dd17041c82 [tenplay] remove extractor(fixes #6927) 2016-03-31 12:02:04 +01:00
fea7295b14 [brightcove] relax embed_in_page regex 2016-03-31 10:48:22 +01:00
9cf01f7f30 [nbc] add new extractor for csnne.com(#5432) 2016-03-31 00:26:42 +01:00
ce548296fe [cnbc] fix test 2016-03-31 00:25:11 +01:00
c02ec7d430 [cnbc] Add new extractor(closes #8012) 2016-03-30 23:18:31 +01:00
6b820a2376 [myspace] improve extraction 2016-03-30 21:18:07 +01:00
e621a344e6 [kwuo] Port to new API and enable --cn-verification-proxy 2016-03-31 02:27:52 +08:00
3ae6f8fec1 [kwuo] Remove _sort_formats() from KuwoBaseIE._get_formats()
Following the idea proposed in 19dbaeece3
2016-03-31 02:11:21 +08:00
597d52fadb [kuwo:song] Correct song ID extraction (fixes #9033)
Bug introduced in daef04a4e7.
2016-03-31 02:00:50 +08:00
afca767d19 [tumblr] Improve _VALID_URL (Closes #9027) 2016-03-30 22:26:43 +06:00
6e359a1534 [comcarcoff] don not depend on crackle extractor(closes #8995)
previously extraction has been delegated to crackle to extract more info
and subtitles #6106 but some of the episodes can't be extracted using
crackle #8995.
2016-03-30 12:27:00 +01:00
6cbb20bb09 Update crunchyroll.py 2016-03-29 14:26:24 -03:00
607619bc90 Add manually generated ISSUE_TEMPLATE.md
In order not to wait for the next release
2016-03-29 22:04:29 +06:00
0b7bfc9422 Improve ISSUE_TEMPLATE_tmpl.md 2016-03-29 22:02:42 +06:00
7168a6c874 [devscripts/make_issue_template] Fix __version__ again 2016-03-29 03:05:15 +06:00
034947dd1e Rename ISSUE_TEMPLATE.tmpl in order not to be picked up by github 2016-03-29 02:48:04 +06:00
3c0de33ad7 Remove ISSUE_TEMPLATE.md 2016-03-29 02:43:48 +06:00
89924f8230 [devscripts/make_issue_template] Fix NameError under python3 2016-03-29 02:41:27 +06:00
a39c68f7e5 Exclude make_issue_template.py from flake8 2016-03-29 02:19:24 +06:00
4a5a67ca25 [devscripts/release.sh] Make ISSUE_TEMPLATE.md and commit it 2016-03-29 02:18:52 +06:00
8751da85a7 [Makefile] Fix ISSUE_TEMPLATE.md target 2016-03-29 02:17:57 +06:00
3bf1df51fd [devscripts/make_issue_template] Rework to use ISSUE_TEMPLATE.tmpl (Closes #8785) 2016-03-29 02:16:38 +06:00
3842a3e652 Add ISSUE_TEMPLATE.tmpl as template for ISSUE_TEMPLATE.md 2016-03-29 02:15:26 +06:00
7710bdf4e8 Add initial ISSUE_TEMPLATE
Add auto-updating of youtube-dl version in ISSUE_TEMPLATE

Move parts of template text and adopt makefile to new format

Moved the 'kind-of-issue' section and rephrased a bit

Rephrased and moved Example URL section upwards

Moved ISSUE_TEMPLATE inside .github folder.

Update makefile to match new folderstructure
2016-03-28 22:43:13 +06:00
8d9dd3c34b [README.md] Add format_id to the list of string meta fields available for use in format selection 2016-03-28 03:08:34 +05:00
33f3040a3e [YoutubeDL] Fix sanitizing subtitles' url 2016-03-28 03:13:39 +06:00
03442072c0 [pornhub] Fix typo (Closes #9008) 2016-03-28 01:21:44 +06:00
c8b13fec02 [foxnews] Restore upload time fields in test 2016-03-28 01:14:12 +06:00
87d105ac6c [amp] Fix upload timestamp extraction (Closes #9007) 2016-03-28 01:13:47 +06:00
3454139576 [pornhub:uservideos] Add support for multipage videos (Closes #9006) 2016-03-28 00:50:46 +06:00
3a23bae9cc [pornhub:playlistbase] Do not include videos not from playlist 2016-03-28 00:32:57 +06:00
8f9a477e7f [pornhub:playlistbase] Use orderedSet 2016-03-28 00:21:08 +06:00
a1cf3e38a3 [bbc] Extend vpid regex (Closes #9003) 2016-03-27 23:22:51 +06:00
a122e7080b release 2016.03.27 2016-03-27 16:56:33 +02:00
b22ca76204 [extractor/common] Filter out unsupported encrypted media for f4m formats (Closes #8573) 2016-03-27 07:42:38 +06:00
f7df343b4a [downloader/f4m] Extract routine for removing unsupported encrypted media 2016-03-27 07:41:19 +06:00
19dbaeece3 Remove _sort_formats from _extract_*_formats methods
Now _sort_formats should be called explicitly.
_sort_formats has been added to all the necessary places in code.

Closes #8051
2016-03-27 07:03:08 +06:00
395fd4b08a [twitter] Handle another form of embedded Vine
Fixes #8996
2016-03-27 04:36:02 +08:00
8018028d0f [pluralsight] Extract chapter metadata (Closes #8993) 2016-03-27 02:10:52 +06:00
00322ad4fd [lynda] Extract chapter metadata (#8993) 2016-03-27 02:00:36 +06:00
4cf3489c6e [vevo] Update videoservice API URL (Closes #8900) 2016-03-27 01:11:11 +06:00
b24ab3e341 [udemy] Improve paid course detection 2016-03-27 00:09:12 +06:00
af4116f4f0 [udemy] Improve format_id 2016-03-27 00:02:52 +06:00
f973e5d54e [udemy] Drop outputs' formats
Always results in 403
2016-03-26 23:55:07 +06:00
62f55aa68a [udemy] Add outputs metadata to view_html formats 2016-03-26 23:54:12 +06:00
02d7634d24 [udemy] Fix outputs' formats format_id 2016-03-26 23:43:25 +06:00
48dce58ca9 [udemy] Use custom sorting 2016-03-26 23:42:46 +06:00
efcba804f6 [udemy] Extract formats from view_html (Closes #8979) 2016-03-26 23:42:34 +06:00
6dee688e6d [youtube:playlistsbase] Restrict playlist regex (Closes #8986) 2016-03-26 20:42:18 +06:00
eedb7ba536 [YoutubeDL] Sort imports 2016-03-26 19:40:33 +06:00
dcf77cf1a7 [YoutubeDL] Sanitize final URLs (Closes #8991) 2016-03-26 19:37:41 +06:00
17bcc626bf [utils] Extract sanitize_url routine 2016-03-26 19:33:57 +06:00
b5a5bbf376 [mailru] Extend _VALID_URL (Closes #8990) 2016-03-26 19:15:32 +06:00
e68d3a010f [twitter] Fix extraction (closes #8966)
HLS and DASH formats are no longer appeared in test cases. I keep them
for fear of triggering new errors.
2016-03-26 18:34:51 +08:00
d10fe8358c [generic] Add a test case for brightcove embed
Closes #8862
2016-03-26 18:30:43 +08:00
d6c340cae5 [brightcove] Extract more formats (#8862) 2016-03-26 18:21:07 +08:00
5964b598ff [brightcove] Support alternative BrightcoveExperience layout
The full URL lays in the `data` attribute of <object> (#8862)
2016-03-26 17:47:32 +08:00
62cdb96f51 release 2016.03.26 2016-03-26 08:58:03 +01:00
e289d6d62c [test_compat] Add tests for compat_urllib_parse_urlencode 2016-03-26 02:38:33 +06:00
6e6bc8dae5 Use urlencode_postdata across the codebase 2016-03-26 02:19:24 +06:00
15707c7e02 [compat] Add compat_urllib_parse_urlencode and eliminate encode_dict
encode_dict functionality has been improved and moved directly into compat_urllib_parse_urlencode
All occurrences of compat_urllib_parse.urlencode throughout the codebase have been replaced by compat_urllib_parse_urlencode

Closes #8974
2016-03-26 01:46:57 +06:00
2156f16ca7 [thescene] Fix extraction and improve style (Closes #8978) 2016-03-25 20:14:34 +06:00
4db441de72 [once] Relax _VALID_URL (Closes #8976) 2016-03-25 19:51:28 +06:00
0be8314dc8 release 2016.03.25 2016-03-25 09:27:18 +01:00
d7f62b049a [iqiyi] Update enc_key 2016-03-25 15:45:40 +08:00
3bb3356812 [douyutv] Extend _VALID_URL 2016-03-25 15:43:29 +08:00
3f15fec1d1 Credit @Kagami for mnet (#8958) 2016-03-25 03:56:27 +06:00
98e68806fb [mnet] Improve (Closes #8958) 2016-03-25 03:26:29 +06:00
e031768666 [mnet] Add new extractor 2016-03-25 02:32:06 +06:00
5eb7db4ee9 [udemy] Add support for new URL schema 2016-03-25 02:28:39 +06:00
f0e83681d9 [udemy] Extract formats from outputs 2016-03-25 02:27:13 +06:00
ff9d5d0938 [udemy] Improve course enrolling 2016-03-25 02:26:46 +06:00
d041a73674 [extractor/__init__] Add youtube:live and sort youtube extractors alphabetically 2016-03-25 01:39:25 +06:00
f07e276a04 [youtube:live] Add extractor (Closes #8959) 2016-03-25 01:18:14 +06:00
993271da0a [nytimes] Tolerate missing metadata (Closes #8952) 2016-03-24 23:28:24 +06:00
369e7e3ff0 [iprima] Fix extraction (Closes #8953) 2016-03-24 22:54:26 +06:00
5767b4eeae [mtv] Fix description extraction (Closes #8962) 2016-03-24 22:23:31 +06:00
622d19160b [utils] Clarify Python versions affected by buggy struct module 2016-03-24 18:06:15 +08:00
32d88410eb [tumblr] Add a test with Instagram embed
Closes #8817
2016-03-24 16:32:53 +08:00
5a51775a58 [generic] Extract Instagram embeds (#8817) 2016-03-24 16:32:27 +08:00
87696e78d7 [instagram] Unescape description (#8817) 2016-03-24 16:30:01 +08:00
c4096e8aea [instagram] Extract embed videos (#8817) 2016-03-24 16:29:33 +08:00
fc27ea9464 [tumblr] Support Vine embeds (#8817) 2016-03-23 23:55:52 +08:00
088e1aac59 [generic] Support Vine embeds (#8817) 2016-03-23 23:55:08 +08:00
81f36eba88 [test/test_utils] Update for escape_url change (again) 2016-03-23 23:23:26 +08:00
2d60465e44 [test/test_utils] Update for escape_url change 2016-03-23 23:20:28 +08:00
4333d56494 Merge pull request #8898 from dstftw/fragment-retries
Add --fragment-retries option (Fixes #8466)
2016-03-23 20:12:32 +05:00
882c699296 [tunein] Fix stream data extraction (Closes #8899, closes #8924) 2016-03-23 20:45:39 +06:00
efbed08dc2 [utils] Encode hostnames before passing to urllib
With IDN (Internationalized Domain Name) and a proxy, non-ascii URLs
are passed down to urllib/urllib2, causing UnicodeEncodeError

Fixes #8890
2016-03-23 22:24:52 +08:00
7da2c87119 Add extractor for thescene.com (closes #8929) 2016-03-22 22:17:59 +01:00
c6ca11f1b3 [once] Prevent ads from embedding into m3u8 playlists (Closes #8893) 2016-03-22 23:48:05 +06:00
2beeb286e1 [laola1tv] Add support for livestreams (Closes #8934) 2016-03-22 22:32:59 +06:00
cc7397b04d [ceskatelevize] Make m3u8 formats extraction non fatal (Closes #8933) 2016-03-22 21:12:29 +06:00
bc5d16b302 [animeondemand] Skip dash for now 2016-03-21 23:37:39 +06:00
85c637b737 [animeondemand] Extract teaser when no full episode available (#8923) 2016-03-21 23:35:50 +06:00
5c69f7a479 [animeondemand] Respect startvideo (Closes #8923) 2016-03-21 23:31:40 +06:00
ff5873b72d [motherless] Detect friends only videos 2016-03-21 22:24:42 +06:00
065c4b27bf [xhamster:embed] Extract vars (Closes #8912) 2016-03-21 22:07:34 +06:00
1600ed1ff9 [rutv] Improve flash version pattern (Closes #8911) 2016-03-21 21:46:49 +06:00
5886b38d73 Add support for https for all extractors as preventive and future-proof measure 2016-03-21 21:36:32 +06:00
0cef27ad25 Add missing r prefix for _VALID_URLs 2016-03-21 21:22:37 +06:00
12af4beb3e [mailru] Add support for https (Closes #8920) 2016-03-21 21:17:29 +06:00
9016d76f71 [YoutubeDL] Improve _format_note 2016-03-20 22:01:45 +06:00
3c5d183c19 [animeondemand] Extract all formats (Closes #8906) 2016-03-20 21:51:22 +06:00
3e8bb9a972 [animeondemand] Detect geo restriction 2016-03-20 20:39:00 +06:00
daef04a4e7 [kwuo] Fix KuwoChartIE and KuwoSingerIE and accept new URL forms 2016-03-20 20:17:56 +08:00
7caae128a7 Credit @vitstradal for the key algorithm in OpenloadIE (#8489)
[ci skip]
2016-03-20 19:12:02 +08:00
2648918c81 [vlive] Fix creator extraction (closes #8814) 2016-03-20 18:15:53 +08:00
920d318d3c README: document that BSD make is also supported (#8902) 2016-03-20 10:55:14 +01:00
9e3c2f1d74 [openload] Misc improvements
* Add thumbnail
* Detect errors (#6469)
* Match more (#6469, #8489)
2016-03-20 16:49:44 +08:00
2bfeee69b9 [openload] Add new extractor (closes #8489) 2016-03-20 15:54:58 +08:00
664bcd80b9 [tudou] Use InAdvancePagedList (closes #8884) 2016-03-20 15:45:31 +08:00
3c20208eff [francetv] Improve formats extraction 2016-03-20 13:00:46 +06:00
db264e3cc3 [francetvinfo] Add support for france3-regions and strip title (Closes #7673) 2016-03-20 12:44:04 +06:00
d396f30467 Merge pull request #8902 from jaimeMF/bmake
Makefile: make it compatible with bmake
2016-03-20 11:08:57 +05:00
96a9f22d98 [discovery] Relax _VALID_URL (Closes #8903) 2016-03-20 10:26:58 +06:00
40025ee2a3 [postprocessort/ffmpeg] Allow embedding webvtt into webm (Closes #8874) 2016-03-20 04:12:34 +06:00
3ff63fb365 Makefile: make it compatible with bmake
It's the portable version of BSD make: http://crufty.net/help/sjg/bmake.html
The syntax for conditionals is different in GNU make and BSD make, so we use the shell
2016-03-19 21:51:13 +01:00
5c7cd37ebd tox.ini: Exclude test_iqiyi_sdk_interpreter.py 2016-03-19 21:50:16 +01:00
298c04b464 [91porn] Use common messages' wording 2016-03-20 02:35:48 +06:00
d95114dd83 [91porn] Unquote final URL (Closes #8881) 2016-03-20 02:34:02 +06:00
94dcade8f8 Credit @jjatria for biobiochiletv (#7314) 2016-03-20 01:36:20 +06:00
fa023ccb2c [biobiochiletv] Fix extraction, extract m3u8 formats and overall improve (Closes #7314) 2016-03-20 01:31:55 +06:00
e36f4aa72b [biobiotv] Add extractor 2016-03-20 01:29:08 +06:00
9261e347cc Credit @kasper93 for cda (#8805) 2016-03-19 23:18:04 +06:00
f1ced6df51 [cda] Improve and simplify (Closes #8805) 2016-03-19 23:17:14 +06:00
8b0d7a66ef [cda] Add new extractor for cda.pl
Fixes #8760
2016-03-19 22:42:40 +06:00
3aec71766d [safari:api] Separate extractor (Closes #8871) 2016-03-19 22:30:48 +06:00
16a8b7986b [downloader/fragment] Document fragment_retries 2016-03-19 20:54:21 +06:00
617e58d850 [downloader/{common,fragment}] Fix total retries reporting on python 2.6 2016-03-19 20:51:30 +06:00
e33baba0dd [downloader/dash] Add fragment retry capability
YouTube may often return 404 HTTP error for a fragment causing the
whole download to fail. However if the same fragment is immediately
retried with the same request data this usually succeeds (1-2 attemps
is usually enough) thus allowing to download the whole file successfully.
So, we will retry all fragments that fail with 404 HTTP error for now.
2016-03-19 20:42:23 +06:00
721f26b821 [downloader/fragment] Add report_retry_fragment 2016-03-19 20:41:24 +06:00
52bb437e41 [options] Add --fragment-retries option 2016-03-19 20:40:36 +06:00
782b1b5bd1 [utils] lookup_unit_table: Match word boundary instead of end of string 2016-03-19 11:44:49 +01:00
0d769bcb78 [extractor/generic] Fix missing byte literal prefix 2016-03-19 05:43:43 +06:00
4cd70099ea [hbo] Add new extractor 2016-03-18 21:18:18 +01:00
09fc33198a utils: lookup_unit_table: Use a stricter regex
In parse_count multiple units start with the same letter, so it would match different units depending on the order they were sorted when iterating over them.
2016-03-18 19:23:06 +01:00
4c3b16d5d1 [test_YoutubeDL] Add test for format_id format selection 2016-03-19 00:04:26 +06:00
d5aacf9a90 Added format_id to the filers on -f. 2016-03-18 23:59:24 +06:00
19e2617a6f [commonprotocols] Add generic support for rtmp URLs (Closes #8488) 2016-03-18 23:42:15 +06:00
edd9b71c2c [extractor/generic] Add a test for m3u playlist served without proper Content-Type 2016-03-18 22:49:11 +06:00
5940862d5a [extractor/generic] Detect m3u playlists served without proper Content-Type 2016-03-18 22:45:28 +06:00
de6c51e88e [extractor/generic] Fix direct link semantics 2016-03-18 22:43:07 +06:00
303dcdb995 [extractor/generic] Simplify upload_date extraction 2016-03-18 22:41:16 +06:00
20938f768b [extractor/generic] Add another test for generic m3u8 2016-03-18 21:54:33 +06:00
955737b2d4 [extractor/generic] Force Content-Type to lowecase 2016-03-18 21:50:44 +06:00
263eff9537 [extractor/generic] Properly extract format id from Content-Type
Fixes extraction for cases like: audio/x-mpegURL; charset=utf-8
2016-03-18 21:50:10 +06:00
cae21032ab [theplatform] Improve geo restriction detection 2016-03-18 21:08:25 +06:00
6187091532 [once] check http formats availability 2016-03-18 11:51:34 +01:00
0d33166ec5 release 2016.03.18 2016-03-18 11:43:48 +01:00
87c03c6bd2 [theplatform] remove unnecessary import 2016-03-18 09:43:28 +01:00
4c92fd2e83 [theplatform] always force theplatform to return a smil for _extract_theplatform_smil 2016-03-18 09:22:10 +01:00
e3d17b3c07 [noz] Fix extraction on python 2.6 by means of using compat_xpath 2016-03-18 02:54:27 +06:00
810c10baa1 [utils] Use compat_xpath 2016-03-18 02:52:23 +06:00
57f7e3c62d [compat] Add compat_xpath 2016-03-18 02:51:38 +06:00
0d0e282912 [animeondemand] Fix typo and improve 2016-03-18 00:13:50 +06:00
85e8f26b82 [animeondemand] Improve extraction 2016-03-18 00:02:34 +06:00
b57fecfddd [animeondemand] Add test 2016-03-17 23:50:10 +06:00
8c97e7efb6 [animeondemand] Expand episode title regex (Closes #8875) 2016-03-17 23:43:14 +06:00
cc162f6a0a [crunchyroll] Fix custom _download_webpage (Closes #8883) 2016-03-17 22:55:04 +06:00
cf45ed786e [wistia] extract more metadata 2016-03-17 17:48:17 +01:00
574b2a7393 [nbc:nbcnews] improve extraction(fixes #6922)
- extract more metadata and formats
- relax regex
2016-03-17 16:11:29 +01:00
9f02ff537c [theplatform] extract brightcove once formats 2016-03-17 16:11:29 +01:00
0436ec0e7a [once] Add new format extractor 2016-03-17 16:11:29 +01:00
11f12195af [youtube] Added itag 91
Seen in https://www.youtube.com/watch?v=jMN4cxyhJjk
2016-03-17 19:25:37 +08:00
a646a8cf98 [sbs] improve extraction(fixes #3811)
- extract error messages
- force the platform smil url(previously the manifest param
in the query is not respected which make theplatform return non working
mp4 files for some videos)
2016-03-17 02:07:06 +01:00
63f41d3821 [bravotv] Add new extractor(#4657) 2016-03-16 21:26:25 +01:00
c5229f3926 [utils] PEP 8 2016-03-16 21:50:04 +06:00
96f4f796fb [brightcover] Remove unused import 2016-03-16 21:47:51 +06:00
70cab344c4 [udemy] Improve course id v4 regex 2016-03-16 21:46:09 +06:00
a7ba57dc17 [udemy] Update course id regex to cover v4 layout (Closes #8753, closes #8868, closes #8870) 2016-03-16 21:45:01 +06:00
83548824c2 Merge pull request #8092 from bpfoley/twitter-thumbnail
[utils] Add extract_attributes for extracting html tag attributes
2016-03-16 13:16:27 +01:00
354dbbd880 [brightcove:new] extract protocol-less embed URLs(closes #2914) 2016-03-16 11:46:53 +01:00
23edc49509 [tv3] Add new extractor(closes #8059) 2016-03-16 10:47:39 +01:00
48254c3f2c [brightcove] some improvements and fixes
- use FFmpeg downloader to download m3u8 formats extracted
from BrightcoveNew(some of the m3u8 media playlists use AES-128)
- update comment and update_url_query to handle url query
2016-03-16 09:21:07 +01:00
2cab48704c [thestar] Add new extractor(closes #5955) 2016-03-15 23:10:31 +01:00
64d4f31d78 [brightcove:new] update embed_in_page embeds regex to match non numeric ref id 2016-03-15 22:50:43 +01:00
0c9ff24041 [noz] fix extraction in python 2.6 2016-03-15 21:00:39 +01:00
3ff8279e80 [kuwo:mv] Fix the test and extraction of georestricted MVs 2016-03-16 02:41:18 +08:00
cb6e477dfe [aljazeera] update the extractor to use BrightcoveNewIE 2016-03-15 19:38:10 +01:00
edfd93518e [svt] extract dashhbbtv formats(#8867) 2016-03-15 19:33:09 +01:00
89807d6a82 [brightcove] extract dash formats and detect audio formats 2016-03-15 18:48:21 +01:00
49dea4913b Merge pull request #8513 from remitamine/dash-sort
[extractor/common] fix dash formats sorting
2016-03-15 18:39:50 +01:00
dec2cae0a7 [twitch:playlistbase] Clarify pagination bug
Pagination bug has been fixed by twitch on 15.03.2016.
2016-03-15 21:45:43 +06:00
cf6cd07396 [noz] extract f4m and m3u8 formats 2016-03-15 15:24:12 +01:00
975b9c9ab0 [brightcove:new] detect m3u8 manifests by M2TS container 2016-03-15 10:06:53 +01:00
8ac73bdbe4 [brightcove:new] Add support for non numeric ref: preffixed video ids 2016-03-15 10:03:08 +01:00
877f440f7b [rice] Add new extractor(closes #1736) 2016-03-15 00:49:23 +01:00
d13bdc3824 [brightcove] raise ExtractorError on 403 errors and fix regex to work with tenplay 2016-03-14 22:24:52 +01:00
744daf9418 [gameinformer] remove unused imports 2016-03-14 21:57:26 +01:00
bf475e1990 [tlc] fix extraction and update extractor to use BrightcoveNewIE 2016-03-14 21:53:00 +01:00
203f3d779a [gameinformer] update the extractor to use BrightcoveNewIE 2016-03-14 18:32:29 +01:00
4230c4894d [external/downloader] fix rtmp downloading using FFmpegFD 2016-03-14 16:51:01 +01:00
6bb266693f release 2016.03.14 2016-03-14 10:25:20 +01:00
5d53c32701 [usatoday] Add new extractor(closes #8655) 2016-03-13 22:36:15 +01:00
2e7e561c1d Merge pull request #8611 from remitamine/ffmpegfd
[downloader/external] Add FFmpegFD
2016-03-13 21:30:27 +01:00
d8515fd41c [downloader/external] pass configuration args to ffmpeg 2016-03-13 21:28:26 +01:00
694c47b261 [external/downloader] don't pass -t and -ss to ffmpeg 2016-03-13 21:28:16 +01:00
77dea16ac8 [downloader/external] check for ffmpeg availablity when it used for m3u8 download 2016-03-13 20:34:51 +01:00
6ae27bed01 [download/external] move the check for multiple selected formats to get_suitable_downloader 2016-03-13 20:34:38 +01:00
da1973a038 [extractor/__init__] disable time range downloading 2016-03-13 16:16:26 +01:00
be24916a7f [downloader/rtsp] Add rtsp and mms downloader 2016-03-13 15:24:02 +01:00
2cb99ebbd0 [downloader/external] add can_download mathod for checking downloader availibilty and support 2016-03-13 15:18:51 +01:00
91ee320bfa [downloader/external] wrap available_opt in a list 2016-03-13 14:37:45 +01:00
8fb754bcd0 Merge pull request #8821 from remitamine/list-thumbnails-order
[YoutubeDL] check for --list-thumbnails immediately after processing them
2016-03-13 12:44:50 +01:00
b7b72db9ad [YoutubeDL] check for --list-thumbnails immediately after processing them 2016-03-13 12:41:15 +01:00
634415ca17 [downloader/external] skip FFmpegFD when requesting multiple formats 2016-03-13 12:23:10 +01:00
2f7ae819ac [utils] PEP 8 2016-03-13 17:23:08 +06:00
0a477f8731 [vice:show] Add extractor (Closes #8847) 2016-03-13 17:22:23 +06:00
a755f82549 [ffmpeg] convert format ext to ffmpeg output formats codes 2016-03-13 12:15:29 +01:00
7f4173ae7c [mixcloud] Fix view count extraction (Closes #8831, closes #8845) 2016-03-13 16:27:58 +06:00
fb47597b09 [bbc] Generalize unit table lookup and add parse_count 2016-03-13 16:27:20 +06:00
450b233cc2 [bbc] Update test 2016-03-13 15:59:54 +06:00
b7d7674f1e [bbc] Update test 2016-03-13 15:56:34 +06:00
0e832c2c97 [bbc] Improve title and description extraction (Closes #8826, closes #8822) 2016-03-13 15:54:56 +06:00
8e4aa7bf18 [bbc] Fix BBC Extractor to work with 'School Report' 2016-03-13 15:54:34 +06:00
a42dfa629e [makerschannel] Add new extractor(closes #8839) 2016-03-12 22:52:53 +01:00
b970dfddaf [minoto] Add new extractor 2016-03-12 22:52:53 +01:00
46a4ea8276 [safari] Remove unused imports 2016-03-13 03:48:38 +06:00
3f2f4a94aa [extractor/generic] Extract f4m formats from final URLs 2016-03-13 03:38:20 +06:00
f930e0c76e [extractor/generic] Extract f4m formats and refactor common info 2016-03-13 03:17:25 +06:00
0fdbb3322b [extractor/common] Add _parse_f4m_formats routine 2016-03-13 03:16:08 +06:00
e9c8999ede [safari] Fix authentication 2016-03-13 02:08:36 +06:00
73cbd709f9 [safari] Respect kaltura session (Closes #7491) 2016-03-13 02:03:07 +06:00
9dce3c095b [kaltura] Respect kaltura session 2016-03-13 02:01:10 +06:00
e5a2e17a9c [kaltura] optimize url info extraction 2016-03-12 18:43:45 +01:00
0ec589fac3 Merge pull request #8827 from remitamine/safari
[safari] extract free and preview videos(#7491)
2016-03-12 17:28:54 +01:00
36bb63e084 [dw] add support for article pages(closes #8790) 2016-03-12 08:33:22 +01:00
91d6aafb48 [dw] add support for audio pages 2016-03-11 23:55:26 +01:00
c8868a9d83 [dw] Add new extractor 2016-03-11 22:44:18 +01:00
09f572fbc0 [extractor/common] add transform_source to _download_smil and _extract_smil_formats 2016-03-11 22:37:07 +01:00
58e6d097d8 [googledrive] Relax _VALID_URL (Closes #8829) 2016-03-12 00:36:39 +06:00
15bf934de5 Merge pull request #8819 from remitamine/simple-webpage-requests
[extractor/common] simplify using data, headers and query params with _download_* methods
2016-03-11 18:19:43 +01:00
cdfee16818 [extractor/common] add data, headers and query params to _request_webpage 2016-03-11 18:12:50 +01:00
bcb668de18 [safari] extract free and preview videos(#7491) 2016-03-11 16:57:06 +01:00
fac7e79277 [kaltura] add support for videos with reference id 2016-03-11 16:52:07 +01:00
a6c8b75904 [common] Use mimeType to determine file extensions (#8766) 2016-03-11 23:51:42 +08:00
25cb05bda9 [utils] Remove codec2ext
This function is orignally used for determining file extensions of DASH
formats. Now in DASH, ext is determined by mime_type. See #8766 for more
information.
2016-03-11 23:51:42 +08:00
6fa6d38549 Credit @benjamincongdon for audioboom (#8812) 2016-03-11 19:46:06 +06:00
883c052378 [audioboom] Improve robustness and extract uploader (Closes #8812) 2016-03-11 19:44:17 +06:00
61f317c24c Added extractor for AudioBoom.com 2016-03-11 19:43:01 +06:00
64f08d4ff2 Merge pull request #8766 from yan12125/dash-detect-ext
Detect file extensions of DASH formats from their codecs
2016-03-11 21:40:07 +08:00
e738e43358 [facebook] Support videos in groups
Viewing/Downloading videos in groups requires logging in, even for
those in public groups.

Fixes #6951.
2016-03-11 16:20:27 +08:00
f6f6217a98 [facebook] Don't override variable in list comprehension 2016-03-10 15:17:04 +01:00
31db8709bf [iqiyi] Update enc_key 2016-03-10 21:37:26 +08:00
5080cbf9fd [facebook] Handle escaped swf params
Fixes #8713
2016-03-10 15:26:32 +08:00
9880124196 [facebook] Fix for m.facebook.com URLs 2016-03-10 14:59:30 +08:00
9c7b509b2a [facebook] Merge FacebookPostIE into FacebookIE
Fixes #8713
2016-03-10 14:59:30 +08:00
e0dccdd398 [test_YoutubeDL] PEP 8 2016-03-10 09:04:48 +06:00
5d583bdf6c [YoutubeDL] Improve _format_note 2016-03-10 01:03:18 +06:00
1e501364d5 [vimeo:ondemand] Clarify IE_NAME 2016-03-10 00:52:52 +06:00
74278def2e [vimeo:ondemand] Separate ondemand extractor (Closes #8330, closes #8801) 2016-03-10 00:51:07 +06:00
e375a149e1 [livestream] Properly build smil URLs (#8794) 2016-03-09 23:11:09 +06:00
2bfc0e97f6 Merge pull request #8791 from benjamincongdon/Twitch-AudioOnly-Rebased
[twitch] Support for "Audio_Only" format
2016-03-08 13:02:56 +06:00
ac45505528 Added flag for 'allow_audio_only' format in Twitch queries 2016-03-07 21:03:24 -06:00
7404061141 Credit @mutantmonkey for ustudio (#8574) and kusi (#8575) 2016-03-07 02:30:47 +06:00
46c329d6f6 [arte] Improve extraction (Closes #8768) 2016-03-07 02:19:54 +06:00
1818e4c2b4 [arte] Fix typo 2016-03-07 02:10:16 +06:00
e7bd17373d [sexu] Improve extraction (Closes #8782) 2016-03-06 18:08:53 +06:00
c58e74062f [Sexu] fix extractor 2016-03-06 17:53:22 +06:00
6d210f2090 [utils] Add more codecs to codec2ext
BBC uses avc3. Here's an example (thanks to @remitamine for this example)

http://rdmedia.bbc.co.uk/dash/ondemand/bbb/2/client_manifest-common_init.mpd

See also https://trac.ffmpeg.org/ticket/5217
2016-03-06 17:57:48 +08:00
af7d5a63b2 [common] Document protocol http_dash_segments 2016-03-06 17:47:07 +08:00
e41acb6364 [safari] Don't pollute std_headers (#8778) 2016-03-06 17:38:39 +08:00
bdf7f13954 release 2016.03.06 2016-03-06 10:08:02 +01:00
0f56a4b443 [vimeo] Don't pollute std_headers
Fixes #8778
2016-03-06 17:01:05 +08:00
1b5284b13f [downloader/fragment] Make speed more smooth
At the beginning of every segment there was a drop to Unknown speed due to timeslice being too small to calculate speed.
Now last speed from the previous fragment is used.
2016-03-06 05:36:52 +06:00
d1e4a464cd [YoutubeDL] Carry long lines and improve readability 2016-03-06 04:32:18 +06:00
ff059017c0 [YoutubeDL] Fix typo in m3u8_native fixup 2016-03-06 04:30:19 +06:00
f22ba4bd60 update tests related to the change in youtube http format sorting
the change was done in 82156fdbf0
2016-03-05 21:52:24 +01:00
1db772673e [cinemassacre] update tests 2016-03-05 21:34:34 +01:00
75313f2baa [cnet] fix info extraction 2016-03-05 21:10:00 +01:00
090eb8e25f Merge pull request #8718 from remitamine/m3u8-fixup
Add fixup for media files produced by HlsNative downloader(fixes #4776)
2016-03-05 18:37:28 +01:00
a9793f58a1 Merge pull request #8754 from remitamine/5min
update 5min related web sites info extraction and add support for Aol features.
2016-03-05 18:35:48 +01:00
7177fd24f8 [vgtv] support ap.vgtv.no and fix old videos extraction(fixes #8719) 2016-03-05 17:51:46 +01:00
1e501f6c40 [jeuxvideo] Fix config URL extraction (Closes #8774) 2016-03-05 21:01:43 +06:00
2629a3802c [revison3] fix video_id for --download-archive 2016-03-05 15:42:15 +01:00
51ce91174b [YoutubeDL] Fix resolution with missing height in output template dict 2016-03-05 19:38:58 +06:00
107d0c421a [revision3] add support for pages of type tag 2016-03-05 13:43:29 +01:00
18b0b23992 [revision3] add support pages of type embed 2016-03-05 12:14:48 +01:00
d1b29d1342 [elpais] Add support for alternative layout (Closes #8744) 2016-03-05 16:43:29 +06:00
2def60c5f3 [common] Use codec2ext for DASH formats (#8764) 2016-03-05 18:18:39 +08:00
19a17d4623 [utils] Add codec2ext 2016-03-05 18:18:28 +08:00
845817aadf [twitter] Provide more metadata 2016-03-05 18:14:58 +08:00
3233a68fbb [utils] update_url_query: Encode the strings in the query dict
The test case with {'test': '第二行тест'} was failing on python 2 (the non-ascii characters were replaced with '?').
2016-03-04 22:18:40 +01:00
cf074e5ddd [foxnews] update test 2016-03-04 21:42:04 +01:00
002c755248 [youporn] Fix sources regex 2016-03-05 01:51:27 +06:00
d627cec608 [youporn] Fix quality extraction (Closes #8758) 2016-03-05 01:50:12 +06:00
1315224cbb [bleacherreport] update tests 2016-03-04 20:14:09 +01:00
7760b9ff4d [audimedia] update _VALID_URL and video_id regex and improve http format_id 2016-03-04 17:55:50 +01:00
28559564b2 [kusi] Correct test_KUSI 2016-03-05 00:04:29 +08:00
fa880d20ad [kusi] Two fixes
Thanks @dstftw for pointing out those
2016-03-04 23:59:58 +08:00
ae7d31af1c [yandexmusic] Capture and output API errors 2016-03-04 21:32:54 +06:00
9d303bf29b Merge branch 'mutantmonkey-kusi' 2016-03-04 23:21:24 +08:00
5f1688f271 [kusi] Simplify and improve 2016-03-04 23:08:47 +08:00
1d4c9ed90c [aol] imporve extraction
- add support for aol features
- remove support for legacy urls
2016-03-04 10:42:58 +01:00
d48352fb5d [engadget] remove support for legacy urls 2016-03-04 10:40:39 +01:00
6d6536acb2 [fivemin] improve extraction
- skip m3u8 formats(404 error)
- skip unavailable test
- download embed page only when it's needed
- update _VALID_URL regex(joystiq.com redirect to engadget.com)
2016-03-04 10:25:16 +01:00
b6f94d81ea [kusi] Add a test for the alternative form of URL 2016-03-04 14:32:01 +08:00
8477a69283 Merge branch 'kusi' of https://github.com/mutantmonkey/youtube-dl into mutantmonkey-kusi 2016-03-04 14:21:23 +08:00
d58cb3ec7e [leeco] Skip an invalid test. test_LePlaylist_1 is sufficient 2016-03-04 13:46:38 +08:00
8a370aedac [leeco] format_id should be strings 2016-03-04 13:38:45 +08:00
24ca0e9c0b [douyutv] Fix tests 2016-03-04 13:36:29 +08:00
e1dd521e49 [livestream] Fix FutureWarning (Closes #8742) 2016-03-04 01:16:58 +06:00
1255733945 Merge pull request #8739 from remitamine/update_url_params
[utils] add update_url_query function to create or update query string params
2016-03-03 19:24:04 +01:00
3201a67f61 [test/test_utils] add more tests for update_url_query 2016-03-03 19:18:57 +01:00
d0ff690d68 [indavideo:embed] Fix tags extraction (Closes #8738) 2016-03-04 00:09:40 +06:00
fb640d0a3d [test/test_utils] add tests for update_url_query 2016-03-03 18:40:05 +01:00
38f9ef31dc [utils] add update_url_query function 2016-03-03 18:34:52 +01:00
a8276b2680 [twitch:playlistbase] Fix all at once fetch 2016-03-03 22:18:32 +06:00
ececca6cde [twitch:playlistbase] Restore original _PAGE_LIMIT 2016-03-03 22:12:55 +06:00
8bbb4b56ee [twitch:playlistsbase] Use orderedSet 2016-03-03 22:11:26 +06:00
539a1641c6 [twitch] Workaround broken paging (Closes #8740) 2016-03-03 22:10:36 +06:00
1b0635aba3 [Makefile] Allow specifying the Python version in offline tests 2016-03-03 21:57:49 +08:00
429491f531 [test/http] Fix failure in Jython
make offlinetest passed on the latest Jython hg version with patched
lib-python/2.7/urllib2.py pulled from CPython 2.7.11
2016-03-03 21:55:17 +08:00
e9c0cdd389 [jython] Introduce compat_os_name
os.name is always 'java' on Jython
2016-03-03 19:24:24 +08:00
0cae023b24 Merge branch 'jython-support'
Closes #8302
2016-03-03 18:49:32 +08:00
8ee239e921 [utils] Jython support - handle filenames correctly
Now test:youtube downloads
2016-03-03 18:47:54 +08:00
8bb56eeeea [utils] Add extract_attributes for extracting html tag attributes
This is much more robust than just using regexps, and handles all
the common scenarios, such as empty/no values, repeated attributes,
entity decoding, mixed case names, and the different possible value
quoting schemes.
2016-03-03 10:11:37 +00:00
fa9e259fd9 [extractor/common] use compat_parse_qs in update_url_params 2016-03-03 10:54:39 +01:00
f3bdae76de [extractor/common] add update_url_params helper method to add or update query string params 2016-03-03 10:27:22 +01:00
03879ff054 [twitter] Media info is not always in the first entity
Fixes #8704
2016-03-03 14:42:49 +08:00
c8398a9b87 [twitter] Now Twitter serves the same file for Firefox and Chrome 2016-03-03 14:27:27 +08:00
b8972bd69d [twitter] Fix extraction of test_Twitter and test_Twitter_1 2016-03-03 14:24:24 +08:00
0ae937a798 [twitter] Support twitter.com/i/videos/tweet/ URLS
Closes #8737
2016-03-03 13:43:45 +08:00
4459bef203 [thepltform] detect other types of errors 2016-03-02 21:41:29 +01:00
e07237f640 [utils] remove check for val from find_xpath_attr 2016-03-02 21:40:21 +01:00
8c5a994424 [leeco] Letv renamed to LeEco
LeEco is the company name and Le is the domain name.

For more information see the Chinese news post
http://www.techorz.com/company-news/letv-renamed-to-leeco-and-new-logo/
2016-03-03 03:27:55 +08:00
2eb25b256b [letv] Merge LetvTvIE into LetvPlaylistIE
And
1. Add more URL examples
2. Improve the matching pattern
2016-03-03 03:27:55 +08:00
f3bc19a989 [letv] Correct regular expressions and fix a typo 2016-03-03 03:27:55 +08:00
7a8fef3173 [letv] Order imports alphabetically 2016-03-03 03:27:55 +08:00
7465e7e42d [letv] Keep videos' order in playlists 2016-03-03 03:27:55 +08:00
5e73a67d44 [letv] Domain name changed 2016-03-03 03:27:55 +08:00
2316dc2b9a [twitch:playlistbase] Mark broken
Twitch paging mechanism is completely broken on twitch side serving all videos all the time and making our travis builds stall.
2016-03-03 00:41:36 +06:00
a2d7797cee [vimeo] Extract uploader_url (Closes #8727) 2016-03-03 00:00:11 +06:00
fd050249af [youtube] Extract uploader_url (Closes #8724) 2016-03-02 23:49:10 +06:00
7bcd2830dd [extractor/common] Document uploader_url 2016-03-02 23:31:24 +06:00
47462a125b [README.md] Document license field for output template 2016-03-02 23:10:01 +06:00
7caf9830b0 [youtube] Extract license (Closes #8725) 2016-03-02 23:07:25 +06:00
2bc0c46f98 [extractor/common] Document license metafield 2016-03-02 23:06:39 +06:00
3318832e9d [youtube] improve width and height extraction from fmt_list 2016-03-02 17:52:13 +01:00
e7d2084568 Merge branch 'master' of github.com:rg3/youtube-dl 2016-03-02 17:35:55 +01:00
c2d3cb4c63 Revert "[youtube] add tbr to _formats extracted from watch_as3.swf"
This reverts commit 4a5ba28a87.
2016-03-02 17:35:04 +01:00
c48dd4400f Revert "[youtube] add basic info for some unknown formats extracted from watch_as3.swf"
This reverts commit 85ca019d96.
2016-03-02 17:34:56 +01:00
e38cafe986 [YoutubeDL] Skip postprocessing and archive report when outputting to stdout (Closes #8729) 2016-03-02 21:11:18 +06:00
85ca019d96 [youtube] add basic info for some unknown formats extracted from watch_as3.swf 2016-03-02 16:05:05 +01:00
4a5ba28a87 [youtube] add tbr to _formats extracted from watch_as3.swf 2016-03-02 16:05:05 +01:00
82156fdbf0 [youtube] extract width and height from fmt_list 2016-03-02 16:05:05 +01:00
6114090418 [nrk:skole] Relax _VALID_URL 2016-03-02 20:57:04 +06:00
3099b31276 [nrk:skole] Add extractor (Closes #8728) 2016-03-02 20:52:06 +06:00
f17f86513e Add fixup for media files produced by HlsNative downloader(fixes #4776) 2016-03-01 21:10:41 +01:00
90f794c6c3 [options] Add --no-mark-watched (#5054) 2016-03-01 23:41:23 +06:00
66ca2cfddd [wistia] Fix extraction (Closes #8707) 2016-03-01 23:26:53 +06:00
269dd2c6a7 Merge pull request #8703 from dstftw/mark-watched
Add --mark-watched feature (Closes #5054)
2016-03-01 23:00:51 +06:00
e7998f59aa [lifenews] Fix extraction and improve (Closes #2482, closes #8714) 2016-03-01 22:59:11 +06:00
9fb556eef0 [iqiyi] SWF URLs are not used anymore
Since automatic detection of enc_key failed

Closes #8705
2016-03-01 08:42:33 +08:00
e781ab63db release 2016.03.01 2016-03-01 00:05:39 +01:00
3e76968220 [rtve.es:live] Fix extraction
* Update _VALID_URL to match the current URLs
* Use the m3u8 manifest since I haven't figured out how to use the rtmp stream
2016-02-29 20:57:26 +01:00
2812c24c16 [mdr] Fix extraction (Closes #8702) 2016-03-01 01:24:26 +06:00
d77ab8e255 Add --mark-watched feature (Closes #5054) 2016-03-01 01:01:33 +06:00
4b3cd7316c [tf1] Improve wat id regex (Closes #8691) 2016-02-29 03:28:21 +06:00
6dae56384a [screenwavemedia] Check formats' URLs 2016-02-28 21:46:36 +06:00
2b2dfae83e [screenwavemedia] Improve formats sorting 2016-02-28 20:16:31 +06:00
6c10dbeae9 [screenwavemedia] Improve formats extraction 2016-02-28 20:05:58 +06:00
9173202b84 [zdf] Ignore hls manifests that use https (closes #8665)
The certificates are misconfigured, you get the following error mesage:

    ssl.CertificateError: hostname u'zdf-hdios-none-i.zdf.de' doesn't match either of 'a248.e.akamai.net', '*.akamaihd.net', '*.akamaihd-staging.net', '*.akamaized.net', '*.akamaized-staging.net'
2016-02-28 14:06:26 +01:00
8870bb4653 [webofstories] Tolerate malforder og:title (Closes #8417) 2016-02-28 03:37:48 +06:00
7a0e7779fe [README.md] Use simple wording instead of env variable for home 2016-02-28 03:12:13 +06:00
a048ffc9b0 [README.md] Clarify configuration file options syntax 2016-02-28 03:04:06 +06:00
4587915b2a [README.md] Make configuration file example more diverse 2016-02-28 02:56:09 +06:00
da665ddc25 release 2016.02.27 2016-02-27 21:31:21 +01:00
5add979d91 [dplay] Add support for dplay.no 2016-02-27 21:42:08 +06:00
20afe8bd14 Credit @aidan- for more dplay sites support (#8463) 2016-02-27 21:31:43 +06:00
940b606a07 [dplay] Improve, extract all formats and metadata (Closes #8463) 2016-02-27 21:30:47 +06:00
9505053704 [dplay] add support for it.dplay.com and dplay.dk 2016-02-27 19:40:36 +06:00
2c9ca78281 [extractor/generic] Add support for tnaflix network embeds (Closes #7505) 2016-02-27 17:15:49 +06:00
63719a8ac3 [tnaflixnetwork:embed] Add _extract_urls 2016-02-27 17:15:06 +06:00
8fab62482a [tnaflixnetwork] Fallback age limit to 18 2016-02-27 16:59:10 +06:00
d6e9c2706f [tnaflixnetwork:embed] Add extractor 2016-02-27 16:58:11 +06:00
f7f2e53a0a [imdb] Recognize 1080p formats (Closes #8677) 2016-02-27 15:51:25 +06:00
9cdffeeb3f [extractor/common] Clarify rationale on media playlist detection 2016-02-27 07:01:11 +06:00
fbb6edd298 [extractor/common] Properly extract audio only formats in master m3u8 playlists 2016-02-27 06:48:13 +06:00
5eb6bdced4 [utils] Multiple changes to base_n()
1. Renamed to encode_base_n()
2. Allow tables longer than 62 characters
3. Raise ValueError instead of AssertionError for invalid input data
4. Return the first character in the table instead of '0' for number 0
5. Add tests
2016-02-27 03:22:52 +08:00
5633b4d39d [infoq] Use BokeCC extractor function 2016-02-27 02:55:11 +08:00
4435c6e98e [bokecc] Add new extractor (#2336) 2016-02-27 02:54:43 +08:00
2ebd2eac88 [letv] Speedup M3U8 decryption 2016-02-27 00:58:03 +08:00
b78b292f0c [youtube] Add alternative automatic captions extraction approach (Closes #8667) 2016-02-26 22:21:47 +06:00
efbd6fb8bb [vidzi] Use decode_packed_codes
Javascript codes found on Vidzi are slightly different from those found
in VideoMega and iQiyi. Nevertheless, the difference has no effects on
the final result.
2016-02-26 15:14:13 +08:00
680079be39 [utils] Relaxing regex in decode_packed_codes for vidzi 2016-02-26 15:13:03 +08:00
e4fc8d2ebe [videomega] Fix extraction (closes #7606) 2016-02-26 15:00:48 +08:00
f52354a889 [utils] Move codes for handling eval() from iqiyi.py 2016-02-26 14:58:29 +08:00
59f898b7a7 [utils] Merge base_n functions 2016-02-26 14:37:20 +08:00
8f4a2124a9 [vidzi] Fix extraction 2016-02-26 14:26:26 +08:00
481888294d [utils] Add base36 for use in Vidzi 2016-02-26 14:26:26 +08:00
d1e440a4a1 [jwplatform] Separate codes for for parsing jwplayer data 2016-02-26 14:26:26 +08:00
81bdc8fdf6 [utils] Move base62 to utils 2016-02-26 14:26:26 +08:00
e048d87fc9 [kuwo] Fix a test 2016-02-26 14:26:26 +08:00
e26cde0927 [space] Remove extractor (Closes #8662)
Now uses ooyala embed
2016-02-25 21:46:43 +06:00
20108c6b90 [ustudio] Improve (Closes #8574) 2016-02-25 21:30:19 +06:00
9195ef745a [uStudio] Add new extractor 2016-02-25 21:29:49 +06:00
d0459c530d [motherless] Update tests 2016-02-25 00:54:41 +06:00
f160785c5c [utils] Remove AM/PM from unified_strdate patterns 2016-02-25 00:52:49 +06:00
5c0a57185c [motherless] Detect non-existing videos 2016-02-25 00:42:19 +06:00
43479d9e9d [motherless] Make categories optional (Closes #8654) 2016-02-25 00:36:14 +06:00
c0da50d2b2 [README.md] Turn references to issues to links 2016-02-24 23:05:23 +06:00
c24883a1c0 [facebook] Fix format sorting
'hd' formats should have higher priorities
2016-02-24 03:43:24 +08:00
1b77ee6248 [c56] Support videos hosted on Sohu (closes #8073) 2016-02-24 03:32:29 +08:00
bf4b3b6bd9 [vk] Extract video URL from extra_data (Closes #8646) 2016-02-23 18:47:13 +06:00
efbeddead3 [facebook] Support mobile URLs (closes #8638) 2016-02-23 13:17:24 +08:00
3cfeb1624a [nba] Support channels (#5362, #4167) 2016-02-23 13:11:20 +08:00
b95dc034ca [utils] Implement cache for OnDemandPagedList 2016-02-23 13:11:20 +08:00
86a7dbe66e [nba] Support non-video/ pages
Fixes #8589
2016-02-23 13:11:20 +08:00
b43a7a92cd [README.md] Fix typo 2016-02-23 05:41:09 +06:00
6563d31710 [README.md] Fix typo 2016-02-23 05:37:49 +06:00
cf89ba9eff [README.md] Clarify robustness and future-proof requirements for new extractors 2016-02-23 05:35:19 +06:00
9b01272832 [README.md] Update link to extractor metafields 2016-02-23 05:03:57 +06:00
58525c94d5 [README.md] Emphasize copyright infringement aspects in add-new-site-support tutorial 2016-02-23 04:58:51 +06:00
621bd0cda9 [README.md] Add tl;dr links to examples 2016-02-23 04:43:45 +06:00
1610f770d7 [README.md] Extract example subsections 2016-02-23 04:29:39 +06:00
0fc871d2f0 [README.md:output_template] Add example for channel/user playlists download 2016-02-23 04:16:47 +06:00
1ad6143061 [xfileshare] Add support for powerwatch (Closes #8628) 2016-02-22 17:37:00 +06:00
92da3cd848 release 2016.02.22 2016-02-22 11:57:31 +01:00
6212bcb191 [tf1] fix info extraction(fixes #8599) 2016-02-22 09:57:40 +01:00
d69abbd3f0 [googledrive] Make thumbnail optional (Closes #8629) 2016-02-22 03:13:18 +06:00
1d00a8823e [arte] PEP 8 2016-02-22 01:32:23 +06:00
5d6e1011df [pbs] Extract all formats (Closes #8538) 2016-02-22 01:23:27 +06:00
f5bdb44443 [extractor/common] Add _remove_duplicate_formats 2016-02-22 01:19:39 +06:00
7efc1c2b49 [twitter] Fix metadata extraction and test_Twitter_1 2016-02-21 17:29:28 +08:00
132e3b74bd [twitter] Fix a typo 2016-02-21 17:21:37 +08:00
bdbf4ba40e [twitter:amplify] Extract more metadata 2016-02-21 17:16:35 +08:00
acb6e97e6a [twitter] Fix several failed tests 2016-02-21 16:57:56 +08:00
445d72b8b5 [twitter:amplify] Add TwitterAmplifyIE for handling Twitter smart URLs
Closes #8075
2016-02-21 16:41:24 +08:00
92c5e11b40 [arte:future] Fix test 2016-02-21 14:23:58 +06:00
0dd046c16c [arte:magazine] Fix test 2016-02-21 13:57:30 +06:00
305168ca3e [arte:+7] Detect more embeds (Closes #8613) 2016-02-21 13:55:25 +06:00
b72f6163dc [arte:+7] Improve _VALID_URL 2016-02-21 13:37:31 +06:00
33d4fdabfa [extractor/generic] Add support for ok embeds (#8619) 2016-02-21 09:51:54 +06:00
cafcf657a4 add more subtitles mime types to mimetype2ext and fix the platform subtitle extraction 2016-02-20 22:02:03 +01:00
101067de12 Jython support - handle *.class files 2016-02-21 03:32:03 +08:00
7360db05b4 [postprocessor/embedthumbnail] Allow mkv to embed thumbnails
Fixes #6046
2016-02-21 03:32:03 +08:00
c1c05c67ea [utils] Jython support - disable setproctitle() until ctypes is complete 2016-02-21 03:32:03 +08:00
399a76e67b [utils] Jython support: tolerate missing fcntl module 2016-02-21 03:32:03 +08:00
765ac263db [utils] mimetype2ext: return 'm4a' for 'audio/mp4' (fixes #8620)
The youtube extractor was using 'mp4' for them, therefore filters like 'bestaudio[ext=m4a]' stopped working (94278f7202 broke it).
2016-02-20 19:55:10 +01:00
a4e4d7dfcd [test_iqiyi_sdk_interpreter] Add test for iQiyi login 2016-02-20 23:10:39 +08:00
73f9c2867d [iqiyi] Support playlists (closes #8019) 2016-02-20 22:44:04 +08:00
9c86d50916 [faz] Future-proof XML element check 2016-02-20 14:11:44 +01:00
1d14c75f55 [Makefile] iQiyi login test requires network 2016-02-20 20:49:30 +08:00
99709cc3f1 [iqiyi] Implement _login()
Currently only email login supported
2016-02-20 19:54:58 +08:00
5bc880b988 [utils] Add OHDave's RSA encryption function 2016-02-20 19:54:58 +08:00
958759f44b [appletrailers] Extend _VALID_URL (#8524) 2016-02-20 15:54:00 +08:00
f34294fa0c [downloader/external:ffmpegfd] check for None value of start_time 2016-02-20 08:06:12 +01:00
99cbe98ce8 [downloader/external] check for external downloaders availability 2016-02-20 07:58:25 +01:00
86bf29050e [test_YoutubeDL] Make test pass until more intelligent sort formats (Closes #8462) 2016-02-20 03:36:03 +06:00
04cbc4980d [mtv] imporove duration extraction 2016-02-19 20:56:45 +01:00
8765151c8a [mtv] Extract duration from each playlist item
RSS used instead of manifest files because it's exact to the millisecond
with the video I tested while in manifest it's only exact to the second.
2016-02-19 19:38:28 +00:00
12b84ac8c1 [downloader/external] Add FFmpegFD(fixes #622)
- replace HlsFD and RtspFD
- add basic support for downloading part of the video or audio
2016-02-19 19:29:24 +01:00
8ec64ac683 [README.md] Clarify verbose log 2016-02-19 22:18:21 +06:00
ed8648a322 [pornhub] Fix thumbnail and duration extraction (Closes #8604) 2016-02-19 21:42:46 +06:00
88641243ab [pornhub:playlistbase] Improve extract entries 2016-02-18 22:30:19 +06:00
40e146aa1e [pornhub:user:videos] Add extractor (Closes #8548) 2016-02-18 22:29:17 +06:00
f3f9cd9234 [francetv] Improve video id regex (Closes #8563) 2016-02-18 22:09:21 +06:00
ebf1b291d0 [youtube:watchlater] Respect --no-playlist 2016-02-18 22:03:46 +06:00
bc7a9cd8fb [youtube:watchlater] Improve _VALID_URL (Closes #8594) 2016-02-18 21:50:21 +06:00
d48502b82a [arte] Improve _VALID_URLs 2016-02-18 21:29:52 +06:00
479ec54a8d [arte:magazine] Improve (Closes #8473) 2016-02-18 21:29:07 +06:00
49625662a9 [arte:magazine] Add extractor 2016-02-18 21:28:18 +06:00
8b809a079a [cbsnews] use find_xpath_attr 2016-02-18 16:10:09 +01:00
778433cb90 [cbsnews] extract subtitle url from theplatform SMIL manifest(fixes #8568) 2016-02-18 15:43:28 +01:00
411cb8f476 [dailymotion] Fix view count extraction
Fix view count parsing when the decimal marker is a whitespace, e.g. '101 101'
2016-02-18 20:31:43 +06:00
63bf4f0dc0 [vrt] Detect geo restriction 2016-02-17 23:28:41 +06:00
80e59a0d5d [vrt] Make formats extraction non fatal (Closes #8587) 2016-02-17 23:18:23 +06:00
8bbd3d1476 [arte] Fix upload date extraction (Closes #8581) 2016-02-17 22:51:08 +06:00
e725e4bced [arte] PEP 8 2016-02-17 22:37:55 +06:00
08d65046f0 [arte] Make sorting aware of en/es formats 2016-02-17 22:37:05 +06:00
44b9745000 [arte] Extend more _VALID_URLs for en and es support 2016-02-17 21:53:53 +06:00
9654fc875b [arte:+7] Fix extraction for react-based layout 2016-02-17 21:49:15 +06:00
0f425e65ec [arte:+7] Add support for en and es URLs 2016-02-17 21:47:18 +06:00
199e724291 [KUSI] Add new extractor 2016-02-16 19:55:46 -08:00
e277f2a63b [orf:tvthek] Check formats (Closes #8580) 2016-02-16 22:23:38 +06:00
f4db09178a [xtube:user] Remove duplicated video ids 2016-02-16 22:06:26 +06:00
86be3cdc2a [xtube] Fix extraction (Closes #8565) 2016-02-16 22:05:23 +06:00
cb64ccc715 [facebook] Improve error handling (#8572) 2016-02-16 09:07:38 +08:00
f66a3c7bc2 [screenjunkies] Fix spelling 2016-02-16 01:30:00 +06:00
fe80df3080 Credit @TingPing for screenjunkies (#8505) 2016-02-16 01:24:57 +06:00
1932476c13 [iqiyi] Omit MD5 sums for the VIP-only video 2016-02-16 02:45:21 +08:00
d2c1f79f20 [youtube:searchurl] Extend _VALID_URL 2016-02-16 00:29:51 +06:00
8eacae8cf9 Credit @RobinHoutevelts for canvas subtiltes (#8537) 2016-02-15 22:33:32 +06:00
c8a80fd818 [screenjunkies] Improve, extract more metadata and workaround subscription (Closes #8505) 2016-02-15 22:29:28 +06:00
b9e8d7140a [screenjunkies] Add new extractor
This doesn't handle the plus only videos yet

Closes #8492
2016-02-15 22:28:36 +06:00
6eff2605d6 [canvas] Add subtitles test (#8537) 2016-02-15 20:59:16 +06:00
fd7a3ea4a4 [canvas] Improve subtitles (Closes #8537) 2016-02-15 20:54:01 +06:00
8d3eeb36d7 [Canvas] Add subtitles 2016-02-15 20:50:03 +06:00
8e0548e180 [iqiyi] Partial support for VIP-only videos
See #8569 and #8019. Currently only 6-min preview are supported
2016-02-15 19:58:24 +08:00
a517bb4b1e [noz] Add new extractor 2016-02-15 00:07:16 +01:00
9dcefb23a1 [laola1tv] Improve (Closes #8478) 2016-02-14 23:40:26 +06:00
d9da74bc06 Credit @blackwinter for laola1tv (#8478) 2016-02-14 23:39:49 +06:00
5e19323ed9 [laola1tv] Fixes for changed site layout.
* Fixed valid URLs (w/ tests).
* Fixed iframe URL extraction.
* Fixed token URL extraction.
* Fixed variable extraction.
* Fixed uploader spelling.
* Added upload_date to result dictionary.
2016-02-14 23:01:49 +06:00
611c1dd96e [refactor] Single quotes consistency 2016-02-14 15:37:17 +06:00
d800609c62 [refactor] Do not specify redundant None as second argument in dict.get() 2016-02-14 14:25:04 +06:00
c78c9cd10d [downloader/dash] PEP 8 2016-02-14 14:13:09 +06:00
e76394f36c [globo] Switch to new-style classes 2016-02-14 14:02:12 +06:00
080e09557d [aes] Switch to new-style classes 2016-02-14 14:01:43 +06:00
fca2e6d5a6 [dailymotion:cloud] Use idiomatic name for classmethod's first argument 2016-02-14 13:44:23 +06:00
b45f2b1d6e [myvideo] Mark broken 2016-02-14 11:24:57 +06:00
fc2e70ee90 Merge pull request #8479 from remitamine/dash_downloader
[downloader/dash] Implement dashsegments fd in terms of fragment fd
2016-02-13 21:12:33 +01:00
b4561e857f [animeondemand] Add .netrc 2016-02-13 22:41:58 +06:00
7023251239 [comedycentral] Support /shows URLs (fixes #8405) 2016-02-13 12:26:27 +01:00
e2bd68c901 [animeondemand][wip] Add extractor (#8518) 2016-02-13 13:30:31 +06:00
dd86780596 [extractor/common] fix dash formats sorting 2016-02-11 10:55:50 +01:00
c43fe0268c [downloader/dash] Implement dashsegments fd in terms of fragment fd 2016-02-09 17:25:44 +01:00
1030 changed files with 127332 additions and 36199 deletions

63
.github/ISSUE_TEMPLATE/1_broken_site.md vendored Normal file
View File

@ -0,0 +1,63 @@
---
name: Broken site support
about: Report broken or misfunctioning site
title: ''
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2021.12.17. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in http://yt-dl.org/escape.
- Search the bugtracker for similar issues: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a broken site support
- [ ] I've verified that I'm running youtube-dl version **2021.12.17**
- [ ] I've checked that all provided URLs are alive and playable in a browser
- [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
- [ ] I've searched the bugtracker for similar issues including closed ones
## Verbose log
<!--
Provide the complete verbose output of youtube-dl that clearly demonstrates the problem.
Add the `-v` flag to your command line you run youtube-dl with (`youtube-dl -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version 2021.12.17
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
[debug] Proxy map: {}
<more lines>
-->
```
PASTE VERBOSE LOG HERE
```
## Description
<!--
Provide an explanation of your issue in an arbitrary form. Provide any additional information, suggested solution and as much context and examples as possible.
If work on your issue requires account credentials please provide them or explain how one can obtain them.
-->
WRITE DESCRIPTION HERE

View File

@ -0,0 +1,54 @@
---
name: Site support request
about: Request support for a new site
title: ''
labels: 'site-support-request'
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2021.12.17. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
- Make sure that site you are requesting is not dedicated to copyright infringement, see https://yt-dl.org/copyright-infringement. youtube-dl does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
- Search the bugtracker for similar site support requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a new site support request
- [ ] I've verified that I'm running youtube-dl version **2021.12.17**
- [ ] I've checked that all provided URLs are alive and playable in a browser
- [ ] I've checked that none of provided URLs violate any copyrights
- [ ] I've searched the bugtracker for similar site support requests including closed ones
## Example URLs
<!--
Provide all kinds of example URLs support for which should be included. Replace following example URLs by yours.
-->
- Single video: https://www.youtube.com/watch?v=BaW_jenozKc
- Single video: https://youtu.be/BaW_jenozKc
- Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc
## Description
<!--
Provide any additional information.
If work on your issue requires account credentials please provide them or explain how one can obtain them.
-->
WRITE DESCRIPTION HERE

View File

@ -0,0 +1,37 @@
---
name: Site feature request
about: Request a new functionality for a site
title: ''
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2021.12.17. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Search the bugtracker for similar site feature requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a site feature request
- [ ] I've verified that I'm running youtube-dl version **2021.12.17**
- [ ] I've searched the bugtracker for similar site feature requests including closed ones
## Description
<!--
Provide an explanation of your site feature request in an arbitrary form. Please make sure the description is worded well enough to be understood, see https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient. Provide any additional information, suggested solution and as much context and examples as possible.
-->
WRITE DESCRIPTION HERE

65
.github/ISSUE_TEMPLATE/4_bug_report.md vendored Normal file
View File

@ -0,0 +1,65 @@
---
name: Bug report
about: Report a bug unrelated to any particular site or extractor
title: ''
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2021.12.17. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in http://yt-dl.org/escape.
- Search the bugtracker for similar issues: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Read bugs section in FAQ: http://yt-dl.org/reporting
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a broken site support issue
- [ ] I've verified that I'm running youtube-dl version **2021.12.17**
- [ ] I've checked that all provided URLs are alive and playable in a browser
- [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
- [ ] I've searched the bugtracker for similar bug reports including closed ones
- [ ] I've read bugs section in FAQ
## Verbose log
<!--
Provide the complete verbose output of youtube-dl that clearly demonstrates the problem.
Add the `-v` flag to your command line you run youtube-dl with (`youtube-dl -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version 2021.12.17
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
[debug] Proxy map: {}
<more lines>
-->
```
PASTE VERBOSE LOG HERE
```
## Description
<!--
Provide an explanation of your issue in an arbitrary form. Please make sure the description is worded well enough to be understood, see https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient. Provide any additional information, suggested solution and as much context and examples as possible.
If work on your issue requires account credentials please provide them or explain how one can obtain them.
-->
WRITE DESCRIPTION HERE

View File

@ -0,0 +1,38 @@
---
name: Feature request
about: Request a new functionality unrelated to any particular site or extractor
title: ''
labels: 'request'
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2021.12.17. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Search the bugtracker for similar feature requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a feature request
- [ ] I've verified that I'm running youtube-dl version **2021.12.17**
- [ ] I've searched the bugtracker for similar feature requests including closed ones
## Description
<!--
Provide an explanation of your issue in an arbitrary form. Please make sure the description is worded well enough to be understood, see https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient. Provide any additional information, suggested solution and as much context and examples as possible.
-->
WRITE DESCRIPTION HERE

38
.github/ISSUE_TEMPLATE/6_question.md vendored Normal file
View File

@ -0,0 +1,38 @@
---
name: Ask question
about: Ask youtube-dl related question
title: ''
labels: 'question'
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- Look through the README (http://yt-dl.org/readme) and FAQ (http://yt-dl.org/faq) for similar questions
- Search the bugtracker for similar questions: http://yt-dl.org/search-issues
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm asking a question
- [ ] I've looked through the README and FAQ for similar questions
- [ ] I've searched the bugtracker for similar questions including closed ones
## Question
<!--
Ask your question in an arbitrary form. Please make sure it's worded well enough to be understood, see https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient.
-->
WRITE QUESTION HERE

1
.github/ISSUE_TEMPLATE/config.yml vendored Normal file
View File

@ -0,0 +1 @@
blank_issues_enabled: false

View File

@ -0,0 +1,63 @@
---
name: Broken site support
about: Report broken or misfunctioning site
title: ''
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is %(version)s. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in http://yt-dl.org/escape.
- Search the bugtracker for similar issues: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a broken site support
- [ ] I've verified that I'm running youtube-dl version **%(version)s**
- [ ] I've checked that all provided URLs are alive and playable in a browser
- [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
- [ ] I've searched the bugtracker for similar issues including closed ones
## Verbose log
<!--
Provide the complete verbose output of youtube-dl that clearly demonstrates the problem.
Add the `-v` flag to your command line you run youtube-dl with (`youtube-dl -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version %(version)s
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
[debug] Proxy map: {}
<more lines>
-->
```
PASTE VERBOSE LOG HERE
```
## Description
<!--
Provide an explanation of your issue in an arbitrary form. Provide any additional information, suggested solution and as much context and examples as possible.
If work on your issue requires account credentials please provide them or explain how one can obtain them.
-->
WRITE DESCRIPTION HERE

View File

@ -0,0 +1,54 @@
---
name: Site support request
about: Request support for a new site
title: ''
labels: 'site-support-request'
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is %(version)s. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
- Make sure that site you are requesting is not dedicated to copyright infringement, see https://yt-dl.org/copyright-infringement. youtube-dl does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
- Search the bugtracker for similar site support requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a new site support request
- [ ] I've verified that I'm running youtube-dl version **%(version)s**
- [ ] I've checked that all provided URLs are alive and playable in a browser
- [ ] I've checked that none of provided URLs violate any copyrights
- [ ] I've searched the bugtracker for similar site support requests including closed ones
## Example URLs
<!--
Provide all kinds of example URLs support for which should be included. Replace following example URLs by yours.
-->
- Single video: https://www.youtube.com/watch?v=BaW_jenozKc
- Single video: https://youtu.be/BaW_jenozKc
- Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc
## Description
<!--
Provide any additional information.
If work on your issue requires account credentials please provide them or explain how one can obtain them.
-->
WRITE DESCRIPTION HERE

View File

@ -0,0 +1,37 @@
---
name: Site feature request
about: Request a new functionality for a site
title: ''
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is %(version)s. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Search the bugtracker for similar site feature requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a site feature request
- [ ] I've verified that I'm running youtube-dl version **%(version)s**
- [ ] I've searched the bugtracker for similar site feature requests including closed ones
## Description
<!--
Provide an explanation of your site feature request in an arbitrary form. Please make sure the description is worded well enough to be understood, see https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient. Provide any additional information, suggested solution and as much context and examples as possible.
-->
WRITE DESCRIPTION HERE

View File

@ -0,0 +1,65 @@
---
name: Bug report
about: Report a bug unrelated to any particular site or extractor
title: ''
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is %(version)s. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in http://yt-dl.org/escape.
- Search the bugtracker for similar issues: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Read bugs section in FAQ: http://yt-dl.org/reporting
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a broken site support issue
- [ ] I've verified that I'm running youtube-dl version **%(version)s**
- [ ] I've checked that all provided URLs are alive and playable in a browser
- [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
- [ ] I've searched the bugtracker for similar bug reports including closed ones
- [ ] I've read bugs section in FAQ
## Verbose log
<!--
Provide the complete verbose output of youtube-dl that clearly demonstrates the problem.
Add the `-v` flag to your command line you run youtube-dl with (`youtube-dl -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version %(version)s
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
[debug] Proxy map: {}
<more lines>
-->
```
PASTE VERBOSE LOG HERE
```
## Description
<!--
Provide an explanation of your issue in an arbitrary form. Please make sure the description is worded well enough to be understood, see https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient. Provide any additional information, suggested solution and as much context and examples as possible.
If work on your issue requires account credentials please provide them or explain how one can obtain them.
-->
WRITE DESCRIPTION HERE

View File

@ -0,0 +1,38 @@
---
name: Feature request
about: Request a new functionality unrelated to any particular site or extractor
title: ''
labels: 'request'
---
<!--
######################################################################
WARNING!
IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################
-->
## Checklist
<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is %(version)s. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
- Search the bugtracker for similar feature requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
- Finally, put x into all relevant boxes (like this [x])
-->
- [ ] I'm reporting a feature request
- [ ] I've verified that I'm running youtube-dl version **%(version)s**
- [ ] I've searched the bugtracker for similar feature requests including closed ones
## Description
<!--
Provide an explanation of your issue in an arbitrary form. Please make sure the description is worded well enough to be understood, see https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient. Provide any additional information, suggested solution and as much context and examples as possible.
-->
WRITE DESCRIPTION HERE

30
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View File

@ -0,0 +1,30 @@
## Please follow the guide below
- You will be asked some questions, please read them **carefully** and answer honestly
- Put an `x` into all the boxes [ ] relevant to your *pull request* (like that [x])
- Use *Preview* tab to see how your *pull request* will actually look like
---
### Before submitting a *pull request* make sure you have:
- [ ] [Searched](https://github.com/ytdl-org/youtube-dl/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
- [ ] Read [adding new extractor tutorial](https://github.com/ytdl-org/youtube-dl#adding-support-for-a-new-site)
- [ ] Read [youtube-dl coding conventions](https://github.com/ytdl-org/youtube-dl#youtube-dl-coding-conventions) and adjusted the code to meet them
- [ ] Covered the code with tests (note that PRs without tests will be REJECTED)
- [ ] Checked the code with [flake8](https://pypi.python.org/pypi/flake8)
### In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check one of the following options:
- [ ] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/)
- [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence)
### What is the purpose of your *pull request*?
- [ ] Bug fix
- [ ] Improvement
- [ ] New extractor
- [ ] New feature
---
### Description of your *pull request* and other information
Explanation of your *pull request* in arbitrary form goes here. Please make sure the description explains the purpose and effect of your *pull request* and is worded well enough to be understood. Provide as much context and examples as possible.

482
.github/workflows/ci.yml vendored Normal file
View File

@ -0,0 +1,482 @@
name: CI
env:
all-cpython-versions: 2.6, 2.7, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 3.10, 3.11, 3.12
main-cpython-versions: 2.7, 3.2, 3.5, 3.9, 3.11
pypy-versions: pypy-2.7, pypy-3.6, pypy-3.7
cpython-versions: main
test-set: core
# Python beta version to be built using pyenv before setup-python support
# Must also be included in all-cpython-versions
next: 3.13
on:
push:
# push inputs aren't known to GitHub
inputs:
cpython-versions:
type: string
default: all
test-set:
type: string
default: core
pull_request:
# pull_request inputs aren't known to GitHub
inputs:
cpython-versions:
type: string
default: main
test-set:
type: string
default: both
workflow_dispatch:
inputs:
cpython-versions:
type: choice
description: CPython versions (main = 2.7, 3.2, 3.5, 3.9, 3.11)
options:
- all
- main
required: true
default: main
test-set:
type: choice
description: core, download
options:
- both
- core
- download
required: true
default: both
permissions:
contents: read
jobs:
select:
name: Select tests from inputs
runs-on: ubuntu-latest
outputs:
cpython-versions: ${{ steps.run.outputs.cpython-versions }}
test-set: ${{ steps.run.outputs.test-set }}
own-pip-versions: ${{ steps.run.outputs.own-pip-versions }}
steps:
# push and pull_request inputs aren't known to GitHub (pt3)
- name: Set push defaults
if: ${{ github.event_name == 'push' }}
env:
cpython-versions: all
test-set: core
run: |
echo "cpython-versions=${{env.cpython-versions}}" >> "$GITHUB_ENV"
echo "test_set=${{env.test_set}}" >> "$GITHUB_ENV"
- name: Get pull_request inputs
if: ${{ github.event_name == 'pull_request' }}
env:
cpython-versions: main
test-set: both
run: |
echo "cpython-versions=${{env.cpython-versions}}" >> "$GITHUB_ENV"
echo "test_set=${{env.test_set}}" >> "$GITHUB_ENV"
- name: Make version array
id: run
run: |
# Make a JSON Array from comma/space-separated string (no extra escaping)
json_list() { \
ret=""; IFS="${IFS},"; set -- $*; \
for a in "$@"; do \
ret=$(printf '%s"%s"' "${ret}${ret:+, }" "$a"); \
done; \
printf '[%s]' "$ret"; }
tests="${{ inputs.test-set || env.test-set }}"
[ $tests = both ] && tests="core download"
printf 'test-set=%s\n' "$(json_list $tests)" >> "$GITHUB_OUTPUT"
versions="${{ inputs.cpython-versions || env.cpython-versions }}"
if [ "$versions" = all ]; then \
versions="${{ env.all-cpython-versions }}"; else \
versions="${{ env.main-cpython-versions }}"; \
fi
printf 'cpython-versions=%s\n' \
"$(json_list ${versions}${versions:+, }${{ env.pypy-versions }})" >> "$GITHUB_OUTPUT"
# versions with a special get-pip.py in a per-version subdirectory
printf 'own-pip-versions=%s\n' \
"$(json_list 2.6, 2.7, 3.2, 3.3, 3.4, 3.5, 3.6)" >> "$GITHUB_OUTPUT"
tests:
name: Run tests
needs: select
permissions:
contents: read
packages: write
runs-on: ${{ matrix.os }}
env:
PIP: python -m pip
PIP_DISABLE_PIP_VERSION_CHECK: true
PIP_NO_PYTHON_VERSION_WARNING: true
strategy:
fail-fast: true
matrix:
os: [ubuntu-22.04]
python-version: ${{ fromJSON(needs.select.outputs.cpython-versions) }}
python-impl: [cpython]
ytdl-test-set: ${{ fromJSON(needs.select.outputs.test-set) }}
run-tests-ext: [sh]
include:
- os: windows-2019
python-version: 3.4
python-impl: cpython
ytdl-test-set: ${{ contains(needs.select.outputs.test-set, 'core') && 'core' || 'nocore' }}
run-tests-ext: bat
- os: windows-2019
python-version: 3.4
python-impl: cpython
ytdl-test-set: ${{ contains(needs.select.outputs.test-set, 'download') && 'download' || 'nodownload' }}
run-tests-ext: bat
# jython
- os: ubuntu-22.04
python-version: 2.7
python-impl: jython
ytdl-test-set: ${{ contains(needs.select.outputs.test-set, 'core') && 'core' || 'nocore' }}
run-tests-ext: sh
- os: ubuntu-22.04
python-version: 2.7
python-impl: jython
ytdl-test-set: ${{ contains(needs.select.outputs.test-set, 'download') && 'download' || 'nodownload' }}
run-tests-ext: sh
steps:
- name: Prepare Linux
if: ${{ startswith(matrix.os, 'ubuntu') }}
shell: bash
run: |
# apt in runner, if needed, may not be up-to-date
sudo apt-get update
- name: Checkout
uses: actions/checkout@v3
#-------- Python 3 -----
- name: Set up supported Python ${{ matrix.python-version }}
id: setup-python
if: ${{ matrix.python-impl == 'cpython' && matrix.python-version != '2.6' && matrix.python-version != '2.7' && matrix.python-version != env.next }}
# wrap broken actions/setup-python@v4
# NB may run apt-get install in Linux
uses: ytdl-org/setup-python@v1
env:
# Temporary (?) workaround for Python 3.5 failures - May 2024
PIP_TRUSTED_HOST: "pypi.python.org pypi.org files.pythonhosted.org"
with:
python-version: ${{ matrix.python-version }}
cache-build: true
allow-build: info
- name: Locate supported Python ${{ matrix.python-version }}
if: ${{ env.pythonLocation }}
shell: bash
run: |
echo "PYTHONHOME=${pythonLocation}" >> "$GITHUB_ENV"
export expected="${{ steps.setup-python.outputs.python-path }}"
dirname() { printf '%s\n' \
'import os, sys' \
'print(os.path.dirname(sys.argv[1]))' \
| ${expected} - "$1"; }
expd="$(dirname "$expected")"
export python="$(command -v python)"
[ "$expd" = "$(dirname "$python")" ] || echo "PATH=$expd:${PATH}" >> "$GITHUB_ENV"
[ -x "$python" ] || printf '%s\n' \
'import os' \
'exp = os.environ["expected"]' \
'python = os.environ["python"]' \
'exps = os.path.split(exp)' \
'if python and (os.path.dirname(python) == exp[0]):' \
' exit(0)' \
'exps[1] = "python" + os.path.splitext(exps[1])[1]' \
'python = os.path.join(*exps)' \
'try:' \
' os.symlink(exp, python)' \
'except AttributeError:' \
' os.rename(exp, python)' \
| ${expected} -
printf '%s\n' \
'import sys' \
'print(sys.path)' \
| ${expected} -
#-------- Python next (was 3.12) -
- name: Set up CPython 3.next environment
if: ${{ matrix.python-impl == 'cpython' && matrix.python-version == env.next }}
shell: bash
run: |
PYENV_ROOT=$HOME/.local/share/pyenv
echo "PYENV_ROOT=${PYENV_ROOT}" >> "$GITHUB_ENV"
- name: Cache Python 3.next
id: cachenext
if: ${{ matrix.python-impl == 'cpython' && matrix.python-version == env.next }}
uses: actions/cache@v3
with:
key: python-${{ env.next }}
path: |
${{ env.PYENV_ROOT }}
- name: Build and set up Python 3.next
if: ${{ matrix.python-impl == 'cpython' && matrix.python-version == env.next && ! steps.cachenext.outputs.cache-hit }}
# dl and build locally
shell: bash
run: |
# Install build environment
sudo apt-get install -y build-essential llvm libssl-dev tk-dev \
libncursesw5-dev libreadline-dev libsqlite3-dev \
libffi-dev xz-utils zlib1g-dev libbz2-dev liblzma-dev
# Download PyEnv from its GitHub repository.
export PYENV_ROOT=${{ env.PYENV_ROOT }}
export PATH=$PYENV_ROOT/bin:$PATH
git clone "https://github.com/pyenv/pyenv.git" "$PYENV_ROOT"
pyenv install ${{ env.next }}
- name: Locate Python 3.next
if: ${{ matrix.python-impl == 'cpython' && matrix.python-version == env.next }}
shell: bash
run: |
PYTHONHOME="$(echo "${{ env.PYENV_ROOT }}/versions/${{ env.next }}."*)"
test -n "$PYTHONHOME"
echo "PYTHONHOME=$PYTHONHOME" >> "$GITHUB_ENV"
echo "PATH=${PYTHONHOME}/bin:$PATH" >> "$GITHUB_ENV"
#-------- Python 2.7 --
- name: Set up Python 2.7
if: ${{ matrix.python-impl == 'cpython' && matrix.python-version == '2.7' }}
# install 2.7
shell: bash
run: |
# Ubuntu 22.04 no longer has python-is-python2: fetch it
curl -L "http://launchpadlibrarian.net/474693132/python-is-python2_2.7.17-4_all.deb" -o python-is-python2.deb
sudo apt-get install -y python2
sudo dpkg --force-breaks -i python-is-python2.deb
echo "PYTHONHOME=/usr" >> "$GITHUB_ENV"
#-------- Python 2.6 --
- name: Set up Python 2.6 environment
if: ${{ matrix.python-impl == 'cpython' && matrix.python-version == '2.6' }}
shell: bash
run: |
openssl_name=openssl-1.0.2u
echo "openssl_name=${openssl_name}" >> "$GITHUB_ENV"
openssl_dir=$HOME/.local/opt/$openssl_name
echo "openssl_dir=${openssl_dir}" >> "$GITHUB_ENV"
PYENV_ROOT=$HOME/.local/share/pyenv
echo "PYENV_ROOT=${PYENV_ROOT}" >> "$GITHUB_ENV"
sudo apt-get install -y openssl ca-certificates
- name: Cache Python 2.6
id: cache26
if: ${{ matrix.python-version == '2.6' }}
uses: actions/cache@v3
with:
key: python-2.6.9
path: |
${{ env.openssl_dir }}
${{ env.PYENV_ROOT }}
- name: Build and set up Python 2.6
if: ${{ matrix.python-impl == 'cpython' && matrix.python-version == '2.6' && ! steps.cache26.outputs.cache-hit }}
# dl and build locally
shell: bash
run: |
# Install build environment
sudo apt-get install -y build-essential llvm libssl-dev tk-dev \
libncursesw5-dev libreadline-dev libsqlite3-dev \
libffi-dev xz-utils zlib1g-dev libbz2-dev liblzma-dev
# Download and install OpenSSL 1.0.2, back in time
openssl_name=${{ env.openssl_name }}
openssl_targz=${openssl_name}.tar.gz
openssl_dir=${{ env.openssl_dir }}
openssl_inc=$openssl_dir/include
openssl_lib=$openssl_dir/lib
openssl_ssl=$openssl_dir/ssl
curl -L "https://www.openssl.org/source/$openssl_targz" -o $openssl_targz
tar -xf $openssl_targz
( cd $openssl_name; \
./config --prefix=$openssl_dir --openssldir=${openssl_dir}/ssl \
--libdir=lib -Wl,-rpath=${openssl_dir}/lib shared zlib-dynamic && \
make && \
make install )
rm -rf $openssl_name
rmdir $openssl_ssl/certs && ln -s /etc/ssl/certs $openssl_ssl/certs
# Download PyEnv from its GitHub repository.
export PYENV_ROOT=${{ env.PYENV_ROOT }}
export PATH=$PYENV_ROOT/bin:$PATH
git clone "https://github.com/pyenv/pyenv.git" "$PYENV_ROOT"
# Prevent pyenv build trying (and failing) to update pip
export GET_PIP=get-pip-2.6.py
echo 'import sys; sys.exit(0)' > ${GET_PIP}
GET_PIP=$(realpath $GET_PIP)
# Build and install Python
export CFLAGS="-I$openssl_inc"
export LDFLAGS="-L$openssl_lib"
export LD_LIBRARY_PATH="$openssl_lib"
pyenv install 2.6.9
- name: Locate Python 2.6
if: ${{ matrix.python-impl == 'cpython' && matrix.python-version == '2.6' }}
shell: bash
run: |
PYTHONHOME="${{ env.PYENV_ROOT }}/versions/2.6.9"
echo "PYTHONHOME=$PYTHONHOME" >> "$GITHUB_ENV"
echo "PATH=${PYTHONHOME}/bin:$PATH" >> "$GITHUB_ENV"
echo "LD_LIBRARY_PATH=${{ env.openssl_dir }}/lib${LD_LIBRARY_PATH:+:}${LD_LIBRARY_PATH}" >> "$GITHUB_ENV"
#-------- Jython ------
- name: Set up Java 8
if: ${{ matrix.python-impl == 'jython' }}
uses: actions/setup-java@v3
with:
java-version: 8
distribution: 'zulu'
- name: Setup Jython environment
if: ${{ matrix.python-impl == 'jython' }}
shell: bash
run: |
echo "JYTHON_ROOT=${HOME}/jython" >> "$GITHUB_ENV"
echo "PIP=pip" >> "$GITHUB_ENV"
- name: Cache Jython
id: cachejy
if: ${{ matrix.python-impl == 'jython' && matrix.python-version == '2.7' }}
uses: actions/cache@v3
with:
# 2.7.3 now available, may solve SNI issue
key: jython-2.7.1
path: |
${{ env.JYTHON_ROOT }}
- name: Install Jython
if: ${{ matrix.python-impl == 'jython' && matrix.python-version == '2.7' && ! steps.cachejy.outputs.cache-hit }}
shell: bash
run: |
JYTHON_ROOT="${{ env.JYTHON_ROOT }}"
curl -L "https://repo1.maven.org/maven2/org/python/jython-installer/2.7.1/jython-installer-2.7.1.jar" -o jython-installer.jar
java -jar jython-installer.jar -s -d "${JYTHON_ROOT}"
echo "${JYTHON_ROOT}/bin" >> "$GITHUB_PATH"
- name: Set up cached Jython
if: ${{ steps.cachejy.outputs.cache-hit }}
shell: bash
run: |
JYTHON_ROOT="${{ env.JYTHON_ROOT }}"
echo "${JYTHON_ROOT}/bin" >> $GITHUB_PATH
- name: Install supporting Python 2.7 if possible
if: ${{ steps.cachejy.outputs.cache-hit }}
shell: bash
run: |
sudo apt-get install -y python2.7 || true
#-------- pip ---------
- name: Set up supported Python ${{ matrix.python-version }} pip
if: ${{ (matrix.python-version != '3.2' && steps.setup-python.outputs.python-path) || matrix.python-version == '2.7' }}
# This step may run in either Linux or Windows
shell: bash
run: |
echo "$PATH"
echo "$PYTHONHOME"
# curl is available on both Windows and Linux, -L follows redirects, -O gets name
python -m ensurepip || python -m pip --version || { \
get_pip="${{ contains(needs.select.outputs.own-pip-versions, matrix.python-version) && format('{0}/', matrix.python-version) || '' }}"; \
curl -L -O "https://bootstrap.pypa.io/pip/${get_pip}get-pip.py"; \
python get-pip.py; }
- name: Set up Python 2.6 pip
if: ${{ matrix.python-version == '2.6' }}
shell: bash
run: |
python -m pip --version || { \
curl -L -O "https://bootstrap.pypa.io/pip/2.6/get-pip.py"; \
curl -L -O "https://files.pythonhosted.org/packages/ac/95/a05b56bb975efa78d3557efa36acaf9cf5d2fd0ee0062060493687432e03/pip-9.0.3-py2.py3-none-any.whl"; \
python get-pip.py --no-setuptools --no-wheel pip-9.0.3-py2.py3-none-any.whl; }
# work-around to invoke pip module on 2.6: https://bugs.python.org/issue2751
echo "PIP=python -m pip.__main__" >> "$GITHUB_ENV"
- name: Set up other Python ${{ matrix.python-version }} pip
if: ${{ matrix.python-version == '3.2' && steps.setup-python.outputs.python-path }}
shell: bash
run: |
python -m pip --version || { \
curl -L -O "https://bootstrap.pypa.io/pip/3.2/get-pip.py"; \
curl -L -O "https://files.pythonhosted.org/packages/b2/d0/cd115fe345dd6f07ec1c780020a7dfe74966fceeb171e0f20d1d4905b0b7/pip-7.1.2-py2.py3-none-any.whl"; \
python get-pip.py --no-setuptools --no-wheel pip-7.1.2-py2.py3-none-any.whl; }
#-------- unittest ----
- name: Upgrade Unittest for Python 2.6
if: ${{ matrix.python-version == '2.6' }}
shell: bash
run: |
# Work around deprecation of support for non-SNI clients at PyPI CDN (see https://status.python.org/incidents/hzmjhqsdjqgb)
$PIP -qq show unittest2 || { \
for u in "65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl" \
"f2/94/3af39d34be01a24a6e65433d19e107099374224905f1e0cc6bbe1fd22a2f/argparse-1.4.0-py2.py3-none-any.whl" \
"c7/a3/c5da2a44c85bfbb6eebcfc1dde24933f8704441b98fdde6528f4831757a6/linecache2-1.0.0-py2.py3-none-any.whl" \
"17/0a/6ac05a3723017a967193456a2efa0aa9ac4b51456891af1e2353bb9de21e/traceback2-1.4.0-py2.py3-none-any.whl" \
"72/20/7f0f433060a962200b7272b8c12ba90ef5b903e218174301d0abfd523813/unittest2-1.1.0-py2.py3-none-any.whl"; do \
curl -L -O "https://files.pythonhosted.org/packages/${u}"; \
$PIP install ${u##*/}; \
done; }
# make tests use unittest2
for test in ./test/test_*.py ./test/helper.py; do
sed -r -i -e '/^import unittest$/s/test/test2 as unittest/' "$test"
done
#-------- nose --------
- name: Install nose for Python ${{ matrix.python-version }}
if: ${{ (matrix.python-version != '3.2' && steps.setup-python.outputs.python-path) || (matrix.python-impl == 'cpython' && (matrix.python-version == '2.7' || matrix.python-version == env.next)) }}
shell: bash
run: |
echo "$PATH"
echo "$PYTHONHOME"
# Use PyNose for recent Pythons instead of Nose
py3ver="${{ matrix.python-version }}"
py3ver=${py3ver#3.}
[ "$py3ver" != "${{ matrix.python-version }}" ] && py3ver=${py3ver%.*} || py3ver=0
[ "$py3ver" -ge 9 ] && nose=pynose || nose=nose
$PIP -qq show $nose || $PIP install $nose
- name: Install nose for other Python 2
if: ${{ matrix.python-impl == 'jython' || (matrix.python-impl == 'cpython' && matrix.python-version == '2.6') }}
shell: bash
run: |
# Work around deprecation of support for non-SNI clients at PyPI CDN (see https://status.python.org/incidents/hzmjhqsdjqgb)
$PIP -qq show nose || { \
curl -L -O "https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl"; \
$PIP install nose-1.3.7-py2-none-any.whl; }
- name: Install nose for other Python 3
if: ${{ matrix.python-version == '3.2' && steps.setup-python.outputs.python-path }}
shell: bash
run: |
$PIP -qq show nose || { \
curl -L -O "https://files.pythonhosted.org/packages/15/d8/dd071918c040f50fa1cf80da16423af51ff8ce4a0f2399b7bf8de45ac3d9/nose-1.3.7-py3-none-any.whl"; \
$PIP install nose-1.3.7-py3-none-any.whl; }
- name: Set up nosetest test
if: ${{ contains(needs.select.outputs.test-set, matrix.ytdl-test-set ) }}
shell: bash
run: |
# set PYTHON_VER
PYTHON_VER=${{ matrix.python-version }}
[ "${PYTHON_VER#*-}" != "$PYTHON_VER" ] || PYTHON_VER="${{ matrix.python-impl }}-${PYTHON_VER}"
echo "PYTHON_VER=$PYTHON_VER" >> "$GITHUB_ENV"
echo "PYTHON_IMPL=${{ matrix.python-impl }}" >> "$GITHUB_ENV"
# define a test to validate the Python version used by nosetests
printf '%s\n' \
'from __future__ import unicode_literals' \
'import sys, os, platform' \
'try:' \
' import unittest2 as unittest' \
'except ImportError:' \
' import unittest' \
'class TestPython(unittest.TestCase):' \
' def setUp(self):' \
' self.ver = os.environ["PYTHON_VER"].split("-")' \
' def test_python_ver(self):' \
' self.assertEqual(["%d" % v for v in sys.version_info[:2]], self.ver[-1].split(".")[:2])' \
' self.assertTrue(sys.version.startswith(self.ver[-1]))' \
' self.assertIn(self.ver[0], ",".join((sys.version, platform.python_implementation())).lower())' \
' def test_python_impl(self):' \
' self.assertIn(platform.python_implementation().lower(), (os.environ["PYTHON_IMPL"], self.ver[0]))' \
> test/test_python.py
#-------- TESTS -------
- name: Run tests
if: ${{ contains(needs.select.outputs.test-set, matrix.ytdl-test-set ) }}
continue-on-error: ${{ matrix.ytdl-test-set == 'download' || matrix.python-impl == 'jython' }}
env:
YTDL_TEST_SET: ${{ matrix.ytdl-test-set }}
run: |
./devscripts/run_tests.${{ matrix.run-tests-ext }}
flake8:
name: Linter
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: 3.9
- name: Install flake8
run: pip install flake8
- name: Run flake8
run: flake8 .

22
.gitignore vendored
View File

@ -1,5 +1,6 @@
*.pyc
*.pyo
*.class
*~
*.DS_Store
wine-py2exe/
@ -12,6 +13,7 @@ README.txt
youtube-dl.1
youtube-dl.bash-completion
youtube-dl.fish
youtube_dl/extractor/lazy_extractors.py
youtube-dl
youtube-dl.exe
youtube-dl.tar.gz
@ -20,16 +22,32 @@ cover/
updates_key.pem
*.egg-info
*.srt
*.ttml
*.sbv
*.vtt
*.flv
*.mp4
*.m4a
*.m4v
*.mp3
*.3gp
*.wav
*.ape
*.mkv
*.swf
*.part
*.ytdl
*.swp
test/testdata
test/local_parameters.json
.tox
youtube-dl.zsh
# IntelliJ related files
.idea
.idea/*
*.iml
tmp/
venv/
# VS Code related files
.vscode

View File

@ -1,19 +0,0 @@
language: python
python:
- "2.6"
- "2.7"
- "3.2"
- "3.3"
- "3.4"
- "3.5"
sudo: false
script: nosetests test --verbose
notifications:
email:
- filippo.valsorda@gmail.com
- phihag@phihag.de
- yasoob.khld@gmail.com
# irc:
# channels:
# - "irc.freenode.org#youtube-dl"
# skip_join: true

92
AUTHORS
View File

@ -26,7 +26,7 @@ Albert Kim
Pierre Rudloff
Huarong Huo
Ismael Mejía
Steffan 'Ruirize' James
Steffan Donal
Andras Elso
Jelle van der Waa
Marcin Cieślak
@ -157,3 +157,93 @@ Founder Fang
Andrew Alexeyew
Saso Bezlaj
Erwin de Haan
Jens Wille
Robin Houtevelts
Patrick Griffis
Aidan Rowe
mutantmonkey
Ben Congdon
Kacper Michajłow
José Joaquín Atria
Viťas Strádal
Kagami Hiiragi
Philip Huppert
blahgeek
Kevin Deldycke
inondle
Tomáš Čech
Déstin Reed
Roman Tsiupa
Artur Krysiak
Jakub Adam Wieczorek
Aleksandar Topuzović
Nehal Patel
Rob van Bekkum
Petr Zvoníček
Pratyush Singh
Aleksander Nitecki
Sebastian Blunt
Matěj Cepl
Xie Yanbo
Philip Xu
John Hawkinson
Rich Leeper
Zhong Jianxin
Thor77
Mattias Wadman
Arjan Verwer
Costy Petrisor
Logan B
Alex Seiler
Vijay Singh
Paul Hartmann
Stephen Chen
Fabian Stahl
Bagira
Odd Stråbø
Philip Herzog
Thomas Christlieb
Marek Rusinowski
Tobias Gruetzmacher
Olivier Bilodeau
Lars Vierbergen
Juanjo Benages
Xiao Di Guan
Thomas Winant
Daniel Twardowski
Jeremie Jarosh
Gerard Rovira
Marvin Ewald
Frédéric Bournival
Timendum
gritstub
Adam Voss
Mike Fährmann
Jan Kundrát
Giuseppe Fabiano
Örn Guðjónsson
Parmjit Virk
Genki Sky
Ľuboš Katrinec
Corey Nicholson
Ashutosh Chaudhary
John Dong
Tatsuyuki Ishi
Daniel Weber
Kay Bouché
Yang Hongbo
Lei Wang
Petr Novák
Leonardo Taccari
Martin Weinelt
Surya Oktafendri
TingPing
Alexandre Macabies
Bastian de Groot
Niklas Haas
András Veres-Szentkirályi
Enes Solak
Nathan Rossi
Thomas van der Berg
Luca Cherubin
Adrian Heine

View File

@ -1,9 +1,9 @@
**Please include the full output of youtube-dl when run with `-v`**, i.e. add `-v` flag to your command line, copy the **whole** output and post it in the issue body wrapped in \`\`\` for better formatting. It should look similar to this:
**Please include the full output of youtube-dl when run with `-v`**, i.e. **add** `-v` flag to **your command line**, copy the **whole** output and post it in the issue body wrapped in \`\`\` for better formatting. It should look similar to this:
```
$ youtube-dl -v http://www.youtube.com/watch?v=BaW_jenozKcj
$ youtube-dl -v <your command line>
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version 2015.12.06
[debug] Git HEAD: 135392e
@ -12,7 +12,7 @@ $ youtube-dl -v http://www.youtube.com/watch?v=BaW_jenozKcj
[debug] Proxy map: {}
...
```
**Do not post screenshots of verbose log only plain text is acceptable.**
**Do not post screenshots of verbose logs; only plain text is acceptable.**
The output (including the first lines) contains important debugging information. Issues without the full output are often not reproducible and therefore do not get solved in short order, if ever.
@ -34,7 +34,7 @@ For bug reports, this means that your report should contain the *complete* outpu
If your server has multiple IPs or you suspect censorship, adding `--call-home` may be a good idea to get more diagnostics. If the error is `ERROR: Unable to extract ...` and you cannot reproduce it from multiple countries, add `--dump-pages` (warning: this will yield a rather large output, redirect it to the file `log.txt` by adding `>log.txt 2>&1` to your command-line) or upload the `.dump` files you get when you add `--write-pages` [somewhere](https://gist.github.com/).
**Site support requests must contain an example URL**. An example URL is a URL you might want to download, like `http://www.youtube.com/watch?v=BaW_jenozKc`. There should be an obvious video present. Except under very special circumstances, the main page of a video service (e.g. `http://www.youtube.com/`) is *not* an example URL.
**Site support requests must contain an example URL**. An example URL is a URL you might want to download, like `https://www.youtube.com/watch?v=BaW_jenozKc`. There should be an obvious video present. Except under very special circumstances, the main page of a video service (e.g. `https://www.youtube.com/`) is *not* an example URL.
### Are you using the latest version?
@ -42,11 +42,11 @@ Before reporting any issue, type `youtube-dl -U`. This should report that you're
### Is the issue already documented?
Make sure that someone has not already opened the issue you're trying to open. Search at the top of the window or browse the [GitHub Issues](https://github.com/rg3/youtube-dl/search?type=Issues) of this repository. If there is an issue, feel free to write something along the lines of "This affects me as well, with version 2015.01.01. Here is some more information on the issue: ...". While some issues may be old, a new post into them often spurs rapid activity.
Make sure that someone has not already opened the issue you're trying to open. Search at the top of the window or browse the [GitHub Issues](https://github.com/ytdl-org/youtube-dl/search?type=Issues) of this repository. If there is an issue, feel free to write something along the lines of "This affects me as well, with version 2015.01.01. Here is some more information on the issue: ...". While some issues may be old, a new post into them often spurs rapid activity.
### Why are existing options not enough?
Before requesting a new feature, please have a quick peek at [the list of supported options](https://github.com/rg3/youtube-dl/blob/master/README.md#synopsis). Many feature requests are for features that actually exist already! Please, absolutely do show off your work in the issue report and detail how the existing similar options do *not* solve your problem.
Before requesting a new feature, please have a quick peek at [the list of supported options](https://github.com/ytdl-org/youtube-dl/blob/master/README.md#options). Many feature requests are for features that actually exist already! Please, absolutely do show off your work in the issue report and detail how the existing similar options do *not* solve your problem.
### Is there enough context in your bug report?
@ -58,7 +58,7 @@ We are then presented with a very complicated request when the original problem
Some of our users seem to think there is a limit of issues they can or should open. There is no limit of issues they can or should open. While it may seem appealing to be able to dump all your issues into one ticket, that means that someone who solves one of your issues cannot mark the issue as closed. Typically, reporting a bunch of issues leads to the ticket lingering since nobody wants to attack that behemoth, until someone mercifully splits the issue into multiple ones.
In particular, every site support request issue should only pertain to services at one site (generally under a common domain, but always using the same backend technology). Do not request support for vimeo user videos, Whitehouse podcasts, and Google Plus pages in the same issue. Also, make sure that you don't post bug reports alongside feature requests. As a rule of thumb, a feature request does not include outputs of youtube-dl that are not immediately related to the feature at hand. Do not post reports of a network error alongside the request for a new video service.
In particular, every site support request issue should only pertain to services at one site (generally under a common domain, but always using the same backend technology). Do not request support for vimeo user videos, White house podcasts, and Google Plus pages in the same issue. Also, make sure that you don't post bug reports alongside feature requests. As a rule of thumb, a feature request does not include outputs of youtube-dl that are not immediately related to the feature at hand. Do not post reports of a network error alongside the request for a new video service.
### Is anyone going to need the feature?
@ -66,11 +66,11 @@ Only post features that you (or an incapacitated friend you can personally talk
### Is your question about youtube-dl?
It may sound strange, but some bug reports we receive are completely unrelated to youtube-dl and relate to a different or even the reporter's own application. Please make sure that you are actually using youtube-dl. If you are using a UI for youtube-dl, report the bug to the maintainer of the actual application providing the UI. On the other hand, if your UI for youtube-dl fails in some way you believe is related to youtube-dl, by all means, go ahead and report the bug.
It may sound strange, but some bug reports we receive are completely unrelated to youtube-dl and relate to a different, or even the reporter's own, application. Please make sure that you are actually using youtube-dl. If you are using a UI for youtube-dl, report the bug to the maintainer of the actual application providing the UI. On the other hand, if your UI for youtube-dl fails in some way you believe is related to youtube-dl, by all means, go ahead and report the bug.
# DEVELOPER INSTRUCTIONS
Most users do not need to build youtube-dl and can [download the builds](http://rg3.github.io/youtube-dl/download.html) or get them from their distribution.
Most users do not need to build youtube-dl and can [download the builds](https://ytdl-org.github.io/youtube-dl/download.html) or get them from their distribution.
To run youtube-dl as a developer, you don't need to build anything either. Simply execute
@ -82,22 +82,34 @@ To run the test, simply invoke your favorite test runner, or execute a test file
python test/test_download.py
nosetests
See item 6 of [new extractor tutorial](#adding-support-for-a-new-site) for how to run extractor specific test cases.
If you want to create a build of youtube-dl yourself, you'll need
* python
* make
* make (only GNU make is supported)
* pandoc
* zip
* nosetests
### Adding support for a new site
If you want to add support for a new site, you can follow this quick list (assuming your service is called `yourextractor`):
If you want to add support for a new site, first of all **make sure** this site is **not dedicated to [copyright infringement](README.md#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free)**. youtube-dl does **not support** such sites thus pull requests adding support for them **will be rejected**.
After you have ensured this site is distributing its content legally, you can follow this quick list (assuming your service is called `yourextractor`):
1. [Fork this repository](https://github.com/ytdl-org/youtube-dl/fork)
2. Check out the source code with:
git clone git@github.com:YOUR_GITHUB_USERNAME/youtube-dl.git
3. Start a new git branch with
cd youtube-dl
git checkout -b yourextractor
1. [Fork this repository](https://github.com/rg3/youtube-dl/fork)
2. Check out the source code with `git clone git@github.com:YOUR_GITHUB_USERNAME/youtube-dl.git`
3. Start a new git branch with `cd youtube-dl; git checkout -b yourextractor`
4. Start with this simple template and save it to `youtube_dl/extractor/yourextractor.py`:
```python
# coding: utf-8
from __future__ import unicode_literals
@ -108,13 +120,13 @@ If you want to add support for a new site, you can follow this quick list (assum
class YourExtractorIE(InfoExtractor):
_VALID_URL = r'https?://(?:www\.)?yourextractor\.com/watch/(?P<id>[0-9]+)'
_TEST = {
'url': 'http://yourextractor.com/watch/42',
'url': 'https://yourextractor.com/watch/42',
'md5': 'TODO: md5 sum of the first 10241 bytes of the video file (use --test)',
'info_dict': {
'id': '42',
'ext': 'mp4',
'title': 'Video title goes here',
'thumbnail': 're:^https?://.*\.jpg$',
'thumbnail': r're:^https?://.*\.jpg$',
# TODO more properties, either as:
# * A value
# * MD5 checksum; start the string with md5:
@ -138,18 +150,285 @@ If you want to add support for a new site, you can follow this quick list (assum
# TODO more properties (see youtube_dl/extractor/common.py)
}
```
5. Add an import in [`youtube_dl/extractor/__init__.py`](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/__init__.py).
6. Run `python test/test_download.py TestDownload.test_YourExtractor`. This *should fail* at first, but you can continually re-run it until you're done. If you decide to add more than one test, then rename ``_TEST`` to ``_TESTS`` and make it into a list of dictionaries. The tests will then be named `TestDownload.test_YourExtractor`, `TestDownload.test_YourExtractor_1`, `TestDownload.test_YourExtractor_2`, etc.
7. Have a look at [`youtube_dl/extractor/common.py`](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/common.py) for possible helper methods and a [detailed description of what your extractor should and may return](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/common.py#L62-L200). Add tests and code for as many as you want.
8. If you can, check the code with [flake8](https://pypi.python.org/pypi/flake8).
9. When the tests pass, [add](http://git-scm.com/docs/git-add) the new files and [commit](http://git-scm.com/docs/git-commit) them and [push](http://git-scm.com/docs/git-push) the result, like this:
5. Add an import in [`youtube_dl/extractor/extractors.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/extractor/extractors.py). This makes the extractor available for use, as long as the class ends with `IE`.
6. Run `python test/test_download.py TestDownload.test_YourExtractor`. This *should fail* at first, but you can continually re-run it until you're done. If you decide to add more than one test, then rename ``_TEST`` to ``_TESTS`` and make it into a list of dictionaries. The tests will then be named `TestDownload.test_YourExtractor`, `TestDownload.test_YourExtractor_1`, `TestDownload.test_YourExtractor_2`, etc. Note that tests with `only_matching` key in test's dict are not counted in.
7. Have a look at [`youtube_dl/extractor/common.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/extractor/common.py) for possible helper methods and a [detailed description of what your extractor should and may return](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303). Add tests and code for as many as you want.
8. Make sure your code follows [youtube-dl coding conventions](#youtube-dl-coding-conventions) and check the code with [flake8](https://flake8.pycqa.org/en/latest/index.html#quickstart):
$ git add youtube_dl/extractor/__init__.py
$ flake8 youtube_dl/extractor/yourextractor.py
9. Make sure your code works under all [Python](https://www.python.org/) versions claimed supported by youtube-dl, namely 2.6, 2.7, and 3.2+.
10. When the tests pass, [add](https://git-scm.com/docs/git-add) the new files and [commit](https://git-scm.com/docs/git-commit) them and [push](https://git-scm.com/docs/git-push) the result, like this:
$ git add youtube_dl/extractor/extractors.py
$ git add youtube_dl/extractor/yourextractor.py
$ git commit -m '[yourextractor] Add new extractor'
$ git push origin yourextractor
10. Finally, [create a pull request](https://help.github.com/articles/creating-a-pull-request). We'll then review and merge it.
11. Finally, [create a pull request](https://help.github.com/articles/creating-a-pull-request). We'll then review and merge it.
In any case, thank you very much for your contributions!
## youtube-dl coding conventions
This section introduces a guide lines for writing idiomatic, robust and future-proof extractor code.
Extractors are very fragile by nature since they depend on the layout of the source data provided by 3rd party media hosters out of your control and this layout tends to change. As an extractor implementer your task is not only to write code that will extract media links and metadata correctly but also to minimize dependency on the source's layout and even to make the code foresee potential future changes and be ready for that. This is important because it will allow the extractor not to break on minor layout changes thus keeping old youtube-dl versions working. Even though this breakage issue is easily fixed by emitting a new version of youtube-dl with a fix incorporated, all the previous versions become broken in all repositories and distros' packages that may not be so prompt in fetching the update from us. Needless to say, some non rolling release distros may never receive an update at all.
### Mandatory and optional metafields
For extraction to work youtube-dl relies on metadata your extractor extracts and provides to youtube-dl expressed by an [information dictionary](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303) or simply *info dict*. Only the following meta fields in the *info dict* are considered mandatory for a successful extraction process by youtube-dl:
- `id` (media identifier)
- `title` (media title)
- `url` (media download URL) or `formats`
In fact only the last option is technically mandatory (i.e. if you can't figure out the download location of the media the extraction does not make any sense). But by convention youtube-dl also treats `id` and `title` as mandatory. Thus the aforementioned metafields are the critical data that the extraction does not make any sense without and if any of them fail to be extracted then the extractor is considered completely broken.
[Any field](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L188-L303) apart from the aforementioned ones are considered **optional**. That means that extraction should be **tolerant** to situations when sources for these fields can potentially be unavailable (even if they are always available at the moment) and **future-proof** in order not to break the extraction of general purpose mandatory fields.
#### Example
Say you have some source dictionary `meta` that you've fetched as JSON with HTTP request and it has a key `summary`:
```python
meta = self._download_json(url, video_id)
```
Assume at this point `meta`'s layout is:
```python
{
...
"summary": "some fancy summary text",
...
}
```
Assume you want to extract `summary` and put it into the resulting info dict as `description`. Since `description` is an optional meta field you should be ready that this key may be missing from the `meta` dict, so that you should extract it like:
```python
description = meta.get('summary') # correct
```
and not like:
```python
description = meta['summary'] # incorrect
```
The latter will break extraction process with `KeyError` if `summary` disappears from `meta` at some later time but with the former approach extraction will just go ahead with `description` set to `None` which is perfectly fine (remember `None` is equivalent to the absence of data).
Similarly, you should pass `fatal=False` when extracting optional data from a webpage with `_search_regex`, `_html_search_regex` or similar methods, for instance:
```python
description = self._search_regex(
r'<span[^>]+id="title"[^>]*>([^<]+)<',
webpage, 'description', fatal=False)
```
With `fatal` set to `False` if `_search_regex` fails to extract `description` it will emit a warning and continue extraction.
You can also pass `default=<some fallback value>`, for example:
```python
description = self._search_regex(
r'<span[^>]+id="title"[^>]*>([^<]+)<',
webpage, 'description', default=None)
```
On failure this code will silently continue the extraction with `description` set to `None`. That is useful for metafields that may or may not be present.
### Provide fallbacks
When extracting metadata try to do so from multiple sources. For example if `title` is present in several places, try extracting from at least some of them. This makes it more future-proof in case some of the sources become unavailable.
#### Example
Say `meta` from the previous example has a `title` and you are about to extract it. Since `title` is a mandatory meta field you should end up with something like:
```python
title = meta['title']
```
If `title` disappears from `meta` in future due to some changes on the hoster's side the extraction would fail since `title` is mandatory. That's expected.
Assume that you have some another source you can extract `title` from, for example `og:title` HTML meta of a `webpage`. In this case you can provide a fallback scenario:
```python
title = meta.get('title') or self._og_search_title(webpage)
```
This code will try to extract from `meta` first and if it fails it will try extracting `og:title` from a `webpage`.
### Regular expressions
#### Don't capture groups you don't use
Capturing group must be an indication that it's used somewhere in the code. Any group that is not used must be non capturing.
##### Example
Don't capture id attribute name here since you can't use it for anything anyway.
Correct:
```python
r'(?:id|ID)=(?P<id>\d+)'
```
Incorrect:
```python
r'(id|ID)=(?P<id>\d+)'
```
#### Make regular expressions relaxed and flexible
When using regular expressions try to write them fuzzy, relaxed and flexible, skipping insignificant parts that are more likely to change, allowing both single and double quotes for quoted values and so on.
##### Example
Say you need to extract `title` from the following HTML code:
```html
<span style="position: absolute; left: 910px; width: 90px; float: right; z-index: 9999;" class="title">some fancy title</span>
```
The code for that task should look similar to:
```python
title = self._search_regex(
r'<span[^>]+class="title"[^>]*>([^<]+)', webpage, 'title')
```
Or even better:
```python
title = self._search_regex(
r'<span[^>]+class=(["\'])title\1[^>]*>(?P<title>[^<]+)',
webpage, 'title', group='title')
```
Note how you tolerate potential changes in the `style` attribute's value or switch from using double quotes to single for `class` attribute:
The code definitely should not look like:
```python
title = self._search_regex(
r'<span style="position: absolute; left: 910px; width: 90px; float: right; z-index: 9999;" class="title">(.*?)</span>',
webpage, 'title', group='title')
```
### Long lines policy
There is a soft limit to keep lines of code under 80 characters long. This means it should be respected if possible and if it does not make readability and code maintenance worse.
For example, you should **never** split long string literals like URLs or some other often copied entities over multiple lines to fit this limit:
Correct:
```python
'https://www.youtube.com/watch?v=FqZTN594JQw&list=PLMYEtVRpaqY00V9W81Cwmzp6N6vZqfUKD4'
```
Incorrect:
```python
'https://www.youtube.com/watch?v=FqZTN594JQw&list='
'PLMYEtVRpaqY00V9W81Cwmzp6N6vZqfUKD4'
```
### Inline values
Extracting variables is acceptable for reducing code duplication and improving readability of complex expressions. However, you should avoid extracting variables used only once and moving them to opposite parts of the extractor file, which makes reading the linear flow difficult.
#### Example
Correct:
```python
title = self._html_search_regex(r'<title>([^<]+)</title>', webpage, 'title')
```
Incorrect:
```python
TITLE_RE = r'<title>([^<]+)</title>'
# ...some lines of code...
title = self._html_search_regex(TITLE_RE, webpage, 'title')
```
### Collapse fallbacks
Multiple fallback values can quickly become unwieldy. Collapse multiple fallback values into a single expression via a list of patterns.
#### Example
Good:
```python
description = self._html_search_meta(
['og:description', 'description', 'twitter:description'],
webpage, 'description', default=None)
```
Unwieldy:
```python
description = (
self._og_search_description(webpage, default=None)
or self._html_search_meta('description', webpage, default=None)
or self._html_search_meta('twitter:description', webpage, default=None))
```
Methods supporting list of patterns are: `_search_regex`, `_html_search_regex`, `_og_search_property`, `_html_search_meta`.
### Trailing parentheses
Always move trailing parentheses after the last argument.
#### Example
Correct:
```python
lambda x: x['ResultSet']['Result'][0]['VideoUrlSet']['VideoUrl'],
list)
```
Incorrect:
```python
lambda x: x['ResultSet']['Result'][0]['VideoUrlSet']['VideoUrl'],
list,
)
```
### Use convenience conversion and parsing functions
Wrap all extracted numeric data into safe functions from [`youtube_dl/utils.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/utils.py): `int_or_none`, `float_or_none`. Use them for string to number conversions as well.
Use `url_or_none` for safe URL processing.
Use `try_get` for safe metadata extraction from parsed JSON.
Use `unified_strdate` for uniform `upload_date` or any `YYYYMMDD` meta field extraction, `unified_timestamp` for uniform `timestamp` extraction, `parse_filesize` for `filesize` extraction, `parse_count` for count meta fields extraction, `parse_resolution`, `parse_duration` for `duration` extraction, `parse_age_limit` for `age_limit` extraction.
Explore [`youtube_dl/utils.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/utils.py) for more useful convenience functions.
#### More examples
##### Safely extract optional description from parsed JSON
```python
description = try_get(response, lambda x: x['result']['video'][0]['summary'], compat_str)
```
##### Safely extract more optional metadata
```python
video = try_get(response, lambda x: x['result']['video'][0], dict) or {}
description = video.get('summary')
duration = float_or_none(video.get('durationMs'), scale=1000)
view_count = int_or_none(video.get('views'))
```

6167
ChangeLog Normal file

File diff suppressed because it is too large Load Diff

View File

@ -1,7 +1,9 @@
include README.md
include test/*.py
include test/*.json
include LICENSE
include AUTHORS
include ChangeLog
include youtube-dl.bash-completion
include youtube-dl.fish
include youtube-dl.1
recursive-include docs Makefile conf.py *.rst
recursive-include test *

View File

@ -1,8 +1,9 @@
all: youtube-dl README.md CONTRIBUTING.md README.txt youtube-dl.1 youtube-dl.bash-completion youtube-dl.zsh youtube-dl.fish supportedsites
clean:
rm -rf youtube-dl.1.temp.md youtube-dl.1 youtube-dl.bash-completion README.txt MANIFEST build/ dist/ .coverage cover/ youtube-dl.tar.gz youtube-dl.zsh youtube-dl.fish *.dump *.part *.info.json *.mp4 *.flv *.mp3 *.avi CONTRIBUTING.md.tmp youtube-dl youtube-dl.exe
rm -rf youtube-dl.1.temp.md youtube-dl.1 youtube-dl.bash-completion README.txt MANIFEST build/ dist/ .coverage cover/ youtube-dl.tar.gz youtube-dl.zsh youtube-dl.fish youtube_dl/extractor/lazy_extractors.py *.dump *.part* *.ytdl *.info.json *.mp4 *.m4a *.flv *.mp3 *.avi *.mkv *.webm *.3gp *.wav *.ape *.swf *.jpg *.png CONTRIBUTING.md.tmp youtube-dl youtube-dl.exe
find . -name "*.pyc" -delete
find . -name "*.class" -delete
PREFIX ?= /usr/local
BINDIR ?= $(PREFIX)/bin
@ -11,15 +12,10 @@ SHAREDIR ?= $(PREFIX)/share
PYTHON ?= /usr/bin/env python
# set SYSCONFDIR to /etc if PREFIX=/usr or PREFIX=/usr/local
ifeq ($(PREFIX),/usr)
SYSCONFDIR=/etc
else
ifeq ($(PREFIX),/usr/local)
SYSCONFDIR=/etc
else
SYSCONFDIR=$(PREFIX)/etc
endif
endif
SYSCONFDIR = $(shell if [ $(PREFIX) = /usr -o $(PREFIX) = /usr/local ]; then echo /etc; else echo $(PREFIX)/etc; fi)
# set markdown input format to "markdown-smart" for pandoc version 2 and to "markdown" for pandoc prior to version 2
MARKDOWN = $(shell if [ `pandoc -v | head -n1 | cut -d" " -f2 | head -c1` = "2" ]; then echo markdown-smart; else echo markdown; fi)
install: youtube-dl youtube-dl.1 youtube-dl.bash-completion youtube-dl.zsh youtube-dl.fish
install -d $(DESTDIR)$(BINDIR)
@ -43,8 +39,17 @@ test:
ot: offlinetest
# Keep this list in sync with devscripts/run_tests.sh
offlinetest: codetest
nosetests --verbose test --exclude test_download.py --exclude test_age_restriction.py --exclude test_subtitles.py --exclude test_write_annotations.py --exclude test_youtube_lists.py
$(PYTHON) -m nose --verbose test \
--exclude test_age_restriction.py \
--exclude test_download.py \
--exclude test_iqiyi_sdk_interpreter.py \
--exclude test_socks.py \
--exclude test_subtitles.py \
--exclude test_write_annotations.py \
--exclude test_youtube_lists.py \
--exclude test_youtube_signature.py
tar: youtube-dl.tar.gz
@ -53,8 +58,15 @@ tar: youtube-dl.tar.gz
pypi-files: youtube-dl.bash-completion README.txt youtube-dl.1 youtube-dl.fish
youtube-dl: youtube_dl/*.py youtube_dl/*/*.py
zip --quiet youtube-dl youtube_dl/*.py youtube_dl/*/*.py
zip --quiet --junk-paths youtube-dl youtube_dl/__main__.py
mkdir -p zip
for d in youtube_dl youtube_dl/downloader youtube_dl/extractor youtube_dl/postprocessor ; do \
mkdir -p zip/$$d ;\
cp -pPR $$d/*.py zip/$$d/ ;\
done
touch -t 200001010101 zip/youtube_dl/*.py zip/youtube_dl/*/*.py
mv zip/youtube_dl/__main__.py zip/
cd zip ; zip -q ../youtube-dl youtube_dl/*.py youtube_dl/*/*.py __main__.py
rm -rf zip
echo '#!$(PYTHON)' > youtube-dl
cat youtube-dl.zip >> youtube-dl
rm youtube-dl.zip
@ -66,15 +78,22 @@ README.md: youtube_dl/*.py youtube_dl/*/*.py
CONTRIBUTING.md: README.md
$(PYTHON) devscripts/make_contributing.py README.md CONTRIBUTING.md
issuetemplates: devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/1_broken_site.md .github/ISSUE_TEMPLATE_tmpl/2_site_support_request.md .github/ISSUE_TEMPLATE_tmpl/3_site_feature_request.md .github/ISSUE_TEMPLATE_tmpl/4_bug_report.md .github/ISSUE_TEMPLATE_tmpl/5_feature_request.md youtube_dl/version.py
$(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/1_broken_site.md .github/ISSUE_TEMPLATE/1_broken_site.md
$(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/2_site_support_request.md .github/ISSUE_TEMPLATE/2_site_support_request.md
$(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/3_site_feature_request.md .github/ISSUE_TEMPLATE/3_site_feature_request.md
$(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/4_bug_report.md .github/ISSUE_TEMPLATE/4_bug_report.md
$(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/5_feature_request.md .github/ISSUE_TEMPLATE/5_feature_request.md
supportedsites:
$(PYTHON) devscripts/make_supportedsites.py docs/supportedsites.md
README.txt: README.md
pandoc -f markdown -t plain README.md -o README.txt
pandoc -f $(MARKDOWN) -t plain README.md -o README.txt
youtube-dl.1: README.md
$(PYTHON) devscripts/prepare_manpage.py >youtube-dl.1.temp.md
pandoc -s -f markdown -t man youtube-dl.1.temp.md -o youtube-dl.1
$(PYTHON) devscripts/prepare_manpage.py youtube-dl.1.temp.md
pandoc -s -f $(MARKDOWN) -t man youtube-dl.1.temp.md -o youtube-dl.1
rm -f youtube-dl.1.temp.md
youtube-dl.bash-completion: youtube_dl/*.py youtube_dl/*/*.py devscripts/bash-completion.in
@ -92,20 +111,25 @@ youtube-dl.fish: youtube_dl/*.py youtube_dl/*/*.py devscripts/fish-completion.in
fish-completion: youtube-dl.fish
youtube-dl.tar.gz: youtube-dl README.md README.txt youtube-dl.1 youtube-dl.bash-completion youtube-dl.zsh youtube-dl.fish
lazy-extractors: youtube_dl/extractor/lazy_extractors.py
_EXTRACTOR_FILES = $(shell find youtube_dl/extractor -iname '*.py' -and -not -iname 'lazy_extractors.py')
youtube_dl/extractor/lazy_extractors.py: devscripts/make_lazy_extractors.py devscripts/lazy_load_template.py $(_EXTRACTOR_FILES)
$(PYTHON) devscripts/make_lazy_extractors.py $@
youtube-dl.tar.gz: youtube-dl README.md README.txt youtube-dl.1 youtube-dl.bash-completion youtube-dl.zsh youtube-dl.fish ChangeLog AUTHORS
@tar -czf youtube-dl.tar.gz --transform "s|^|youtube-dl/|" --owner 0 --group 0 \
--exclude '*.DS_Store' \
--exclude '*.kate-swp' \
--exclude '*.pyc' \
--exclude '*.pyo' \
--exclude '*~' \
--exclude '__pycache' \
--exclude '__pycache__' \
--exclude '.git' \
--exclude 'testdata' \
--exclude 'docs/_build' \
-- \
bin devscripts test youtube_dl docs \
LICENSE README.md README.txt \
ChangeLog AUTHORS LICENSE README.md README.txt \
Makefile MANIFEST.in youtube-dl.1 youtube-dl.bash-completion \
youtube-dl.zsh youtube-dl.fish setup.py \
youtube-dl.zsh youtube-dl.fish setup.py setup.cfg \
youtube-dl

1558
README.md

File diff suppressed because it is too large Load Diff

1
devscripts/__init__.py Normal file
View File

@ -0,0 +1 @@
# Empty file needed to make devscripts.utils properly importable from outside

View File

@ -5,8 +5,12 @@ import os
from os.path import dirname as dirn
import sys
sys.path.insert(0, dirn(dirn((os.path.abspath(__file__)))))
sys.path.insert(0, dirn(dirn(os.path.abspath(__file__))))
import youtube_dl
from youtube_dl.compat import compat_open as open
from utils import read_file
BASH_COMPLETION_FILE = "youtube-dl.bash-completion"
BASH_COMPLETION_TEMPLATE = "devscripts/bash-completion.in"
@ -18,12 +22,12 @@ def build_completion(opt_parser):
for option in group.option_list:
# for every long flag
opts_flag.append(option.get_opt_string())
with open(BASH_COMPLETION_TEMPLATE) as f:
template = f.read()
with open(BASH_COMPLETION_FILE, "w") as f:
template = read_file(BASH_COMPLETION_TEMPLATE)
with open(BASH_COMPLETION_FILE, "w", encoding='utf-8') as f:
# just using the special char
filled_template = template.replace("{{flags}}", " ".join(opts_flag))
f.write(filled_template)
parser = youtube_dl.parseOpts()[0]
build_completion(parser)

View File

@ -1,17 +1,38 @@
#!/usr/bin/python3
from http.server import HTTPServer, BaseHTTPRequestHandler
from socketserver import ThreadingMixIn
import argparse
import ctypes
import functools
import shutil
import subprocess
import sys
import tempfile
import threading
import traceback
import os.path
sys.path.insert(0, os.path.dirname(os.path.dirname((os.path.abspath(__file__)))))
from youtube_dl.compat import (
compat_input,
compat_http_server,
compat_str,
compat_urlparse,
)
class BuildHTTPServer(ThreadingMixIn, HTTPServer):
# These are not used outside of buildserver.py thus not in compat.py
try:
import winreg as compat_winreg
except ImportError: # Python 2
import _winreg as compat_winreg
try:
import socketserver as compat_socketserver
except ImportError: # Python 2
import SocketServer as compat_socketserver
class BuildHTTPServer(compat_socketserver.ThreadingMixIn, compat_http_server.HTTPServer):
allow_reuse_address = True
@ -191,7 +212,7 @@ def main(args=None):
action='store_const', dest='action', const='service',
help='Run as a Windows service')
parser.add_argument('-b', '--bind', metavar='<host:port>',
action='store', default='localhost:8142',
action='store', default='0.0.0.0:8142',
help='Bind to host:port (default %default)')
options = parser.parse_args(args=args)
@ -216,7 +237,7 @@ def main(args=None):
srv = BuildHTTPServer((host, port), BuildHTTPRequestHandler)
thr = threading.Thread(target=srv.serve_forever)
thr.start()
input('Press ENTER to shut down')
compat_input('Press ENTER to shut down')
srv.shutdown()
thr.join()
@ -231,8 +252,6 @@ def rmtree(path):
os.remove(fname)
os.rmdir(path)
#==============================================================================
class BuildError(Exception):
def __init__(self, output, code=500):
@ -249,15 +268,25 @@ class HTTPError(BuildError):
class PythonBuilder(object):
def __init__(self, **kwargs):
pythonVersion = kwargs.pop('python', '2.7')
try:
key = _winreg.OpenKey(_winreg.HKEY_LOCAL_MACHINE, r'SOFTWARE\Python\PythonCore\%s\InstallPath' % pythonVersion)
python_version = kwargs.pop('python', '3.4')
python_path = None
for node in ('Wow6432Node\\', ''):
try:
self.pythonPath, _ = _winreg.QueryValueEx(key, '')
finally:
_winreg.CloseKey(key)
except Exception:
raise BuildError('No such Python version: %s' % pythonVersion)
key = compat_winreg.OpenKey(
compat_winreg.HKEY_LOCAL_MACHINE,
r'SOFTWARE\%sPython\PythonCore\%s\InstallPath' % (node, python_version))
try:
python_path, _ = compat_winreg.QueryValueEx(key, '')
finally:
compat_winreg.CloseKey(key)
break
except Exception:
pass
if not python_path:
raise BuildError('No such Python version: %s' % python_version)
self.pythonPath = python_path
super(PythonBuilder, self).__init__(**kwargs)
@ -293,7 +322,7 @@ class GITBuilder(GITInfoBuilder):
class YoutubeDLBuilder(object):
authorizedUsers = ['fraca7', 'phihag', 'rg3', 'FiloSottile']
authorizedUsers = ['fraca7', 'phihag', 'rg3', 'FiloSottile', 'ytdl-org']
def __init__(self, **kwargs):
if self.repoName != 'youtube-dl':
@ -305,8 +334,10 @@ class YoutubeDLBuilder(object):
def build(self):
try:
subprocess.check_output([os.path.join(self.pythonPath, 'python.exe'), 'setup.py', 'py2exe'],
cwd=self.buildPath)
proc = subprocess.Popen([os.path.join(self.pythonPath, 'python.exe'), 'setup.py', 'py2exe'], stdin=subprocess.PIPE, cwd=self.buildPath)
proc.wait()
#subprocess.check_output([os.path.join(self.pythonPath, 'python.exe'), 'setup.py', 'py2exe'],
# cwd=self.buildPath)
except subprocess.CalledProcessError as e:
raise BuildError(e.output)
@ -369,12 +400,12 @@ class Builder(PythonBuilder, GITBuilder, YoutubeDLBuilder, DownloadBuilder, Clea
pass
class BuildHTTPRequestHandler(BaseHTTPRequestHandler):
class BuildHTTPRequestHandler(compat_http_server.BaseHTTPRequestHandler):
actionDict = {'build': Builder, 'download': Builder} # They're the same, no more caching.
def do_GET(self):
path = urlparse.urlparse(self.path)
paramDict = dict([(key, value[0]) for key, value in urlparse.parse_qs(path.query).items()])
path = compat_urlparse.urlparse(self.path)
paramDict = dict([(key, value[0]) for key, value in compat_urlparse.parse_qs(path.query).items()])
action, _, path = path.path.strip('/').partition('/')
if path:
path = path.split('/')
@ -388,19 +419,15 @@ class BuildHTTPRequestHandler(BaseHTTPRequestHandler):
builder.close()
except BuildError as e:
self.send_response(e.code)
msg = unicode(e).encode('UTF-8')
msg = compat_str(e).encode('UTF-8')
self.send_header('Content-Type', 'text/plain; charset=UTF-8')
self.send_header('Content-Length', len(msg))
self.end_headers()
self.wfile.write(msg)
except HTTPError as e:
self.send_response(e.code, str(e))
else:
self.send_response(500, 'Unknown build method "%s"' % action)
else:
self.send_response(500, 'Malformed URL')
#==============================================================================
if __name__ == '__main__':
main()

View File

@ -14,7 +14,7 @@ import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from test.helper import get_testcases
from test.helper import gettestcases
from youtube_dl.utils import compat_urllib_parse_urlparse
from youtube_dl.utils import compat_urllib_request
@ -24,7 +24,7 @@ if len(sys.argv) > 1:
else:
METHOD = 'EURISTIC'
for test in get_testcases():
for test in gettestcases():
if METHOD == 'EURISTIC':
try:
webpage = compat_urllib_request.urlopen(test['url'], timeout=10).read()
@ -45,12 +45,12 @@ for test in get_testcases():
RESULT = ('.' + domain + '\n' in LIST or '\n' + domain + '\n' in LIST)
if RESULT and ('info_dict' not in test or 'age_limit' not in test['info_dict'] or
test['info_dict']['age_limit'] != 18):
if RESULT and ('info_dict' not in test or 'age_limit' not in test['info_dict']
or test['info_dict']['age_limit'] != 18):
print('\nPotential missing age_limit check: {0}'.format(test['name']))
elif not RESULT and ('info_dict' in test and 'age_limit' in test['info_dict'] and
test['info_dict']['age_limit'] == 18):
elif not RESULT and ('info_dict' in test and 'age_limit' in test['info_dict']
and test['info_dict']['age_limit'] == 18):
print('\nPotential false negative: {0}'.format(test['name']))
else:

83
devscripts/cli_to_api.py Executable file
View File

@ -0,0 +1,83 @@
#!/usr/bin/env python
# coding: utf-8
from __future__ import unicode_literals
"""
This script displays the API parameters corresponding to a yt-dl command line
Example:
$ ./cli_to_api.py -f best
{u'format': 'best'}
$
"""
# Allow direct execution
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import youtube_dl
from types import MethodType
def cli_to_api(*opts):
YDL = youtube_dl.YoutubeDL
# to extract the parsed options, break out of YoutubeDL instantiation
# return options via this Exception
class ParseYTDLResult(Exception):
def __init__(self, result):
super(ParseYTDLResult, self).__init__('result')
self.opts = result
# replacement constructor that raises ParseYTDLResult
def ytdl_init(ydl, ydl_opts):
super(YDL, ydl).__init__(ydl_opts)
raise ParseYTDLResult(ydl_opts)
# patch in the constructor
YDL.__init__ = MethodType(ytdl_init, YDL)
# core parser
def parsed_options(argv):
try:
youtube_dl._real_main(list(argv))
except ParseYTDLResult as result:
return result.opts
# from https://github.com/yt-dlp/yt-dlp/issues/5859#issuecomment-1363938900
default = parsed_options([])
def neq_opt(a, b):
if a == b:
return False
if a is None and repr(type(object)).endswith(".utils.DateRange'>"):
return '0001-01-01 - 9999-12-31' != '{0}'.format(b)
return a != b
diff = dict((k, v) for k, v in parsed_options(opts).items() if neq_opt(default[k], v))
if 'postprocessors' in diff:
diff['postprocessors'] = [pp for pp in diff['postprocessors'] if pp not in default['postprocessors']]
return diff
def main():
from pprint import PrettyPrinter
pprint = PrettyPrinter()
super_format = pprint.format
def format(object, context, maxlevels, level):
if repr(type(object)).endswith(".utils.DateRange'>"):
return '{0}: {1}>'.format(repr(object)[:-2], object), True, False
return super_format(object, context, maxlevels, level)
pprint.format = format
pprint.pprint(cli_to_api(*sys.argv))
if __name__ == '__main__':
main()

View File

@ -0,0 +1,111 @@
#!/usr/bin/env python
from __future__ import unicode_literals
import json
import mimetypes
import netrc
import optparse
import os
import re
import sys
dirn = os.path.dirname
sys.path.insert(0, dirn(dirn(os.path.abspath(__file__))))
from youtube_dl.compat import (
compat_basestring,
compat_getpass,
compat_print,
compat_urllib_request,
)
from youtube_dl.utils import (
make_HTTPS_handler,
sanitized_Request,
)
from utils import read_file
class GitHubReleaser(object):
_API_URL = 'https://api.github.com/repos/ytdl-org/youtube-dl/releases'
_UPLOADS_URL = 'https://uploads.github.com/repos/ytdl-org/youtube-dl/releases/%s/assets?name=%s'
_NETRC_MACHINE = 'github.com'
def __init__(self, debuglevel=0):
self._init_github_account()
https_handler = make_HTTPS_handler({}, debuglevel=debuglevel)
self._opener = compat_urllib_request.build_opener(https_handler)
def _init_github_account(self):
try:
info = netrc.netrc().authenticators(self._NETRC_MACHINE)
if info is not None:
self._token = info[2]
compat_print('Using GitHub credentials found in .netrc...')
return
else:
compat_print('No GitHub credentials found in .netrc')
except (IOError, netrc.NetrcParseError):
compat_print('Unable to parse .netrc')
self._token = compat_getpass(
'Type your GitHub PAT (personal access token) and press [Return]: ')
def _call(self, req):
if isinstance(req, compat_basestring):
req = sanitized_Request(req)
req.add_header('Authorization', 'token %s' % self._token)
response = self._opener.open(req).read().decode('utf-8')
return json.loads(response)
def list_releases(self):
return self._call(self._API_URL)
def create_release(self, tag_name, name=None, body='', draft=False, prerelease=False):
data = {
'tag_name': tag_name,
'target_commitish': 'master',
'name': name,
'body': body,
'draft': draft,
'prerelease': prerelease,
}
req = sanitized_Request(self._API_URL, json.dumps(data).encode('utf-8'))
return self._call(req)
def create_asset(self, release_id, asset):
asset_name = os.path.basename(asset)
url = self._UPLOADS_URL % (release_id, asset_name)
# Our files are small enough to be loaded directly into memory.
data = open(asset, 'rb').read()
req = sanitized_Request(url, data)
mime_type, _ = mimetypes.guess_type(asset_name)
req.add_header('Content-Type', mime_type or 'application/octet-stream')
return self._call(req)
def main():
parser = optparse.OptionParser(usage='%prog CHANGELOG VERSION BUILDPATH')
options, args = parser.parse_args()
if len(args) != 3:
parser.error('Expected a version and a build directory')
changelog_file, version, build_path = args
changelog = read_file(changelog_file)
mobj = re.search(r'(?s)version %s\n{2}(.+?)\n{3}' % version, changelog)
body = mobj.group(1) if mobj else ''
releaser = GitHubReleaser()
new_release = releaser.create_release(
version, name='youtube-dl %s' % version, body=body)
release_id = new_release['id']
for asset in os.listdir(build_path):
compat_print('Uploading %s...' % asset)
releaser.create_asset(release_id, os.path.join(build_path, asset))
if __name__ == '__main__':
main()

View File

@ -6,10 +6,13 @@ import os
from os.path import dirname as dirn
import sys
sys.path.insert(0, dirn(dirn((os.path.abspath(__file__)))))
sys.path.insert(0, dirn(dirn(os.path.abspath(__file__))))
import youtube_dl
from youtube_dl.utils import shell_quote
from utils import read_file, write_file
FISH_COMPLETION_FILE = 'youtube-dl.fish'
FISH_COMPLETION_TEMPLATE = 'devscripts/fish-completion.in'
@ -38,11 +41,10 @@ def build_completion(opt_parser):
complete_cmd.extend(EXTRA_ARGS.get(long_option, []))
commands.append(shell_quote(complete_cmd))
with open(FISH_COMPLETION_TEMPLATE) as f:
template = f.read()
template = read_file(FISH_COMPLETION_TEMPLATE)
filled_template = template.replace('{{commands}}', '\n'.join(commands))
with open(FISH_COMPLETION_FILE, 'w') as f:
f.write(filled_template)
write_file(FISH_COMPLETION_FILE, filled_template)
parser = youtube_dl.parseOpts()[0]
build_completion(parser)

View File

@ -23,6 +23,7 @@ def openssl_encode(algo, key, iv):
out, _ = prog.communicate(secret_msg)
return out
iv = key = [0x20, 0x15] + 14 * [0]
r = openssl_encode('aes-128-cbc', key, iv)

View File

@ -6,16 +6,21 @@ import sys
import hashlib
import os.path
dirn = os.path.dirname
sys.path.insert(0, dirn(dirn(dirn(os.path.abspath(__file__)))))
from devscripts.utils import read_file, write_file
from youtube_dl.compat import compat_open as open
if len(sys.argv) <= 1:
print('Specify the version number as parameter')
sys.exit()
version = sys.argv[1]
with open('update/LATEST_VERSION', 'w') as f:
f.write(version)
write_file('update/LATEST_VERSION', version)
versions_info = json.load(open('update/versions.json'))
versions_info = json.loads(read_file('update/versions.json'))
if 'signature' in versions_info:
del versions_info['signature']
@ -39,5 +44,5 @@ for key, filename in filenames.items():
versions_info['versions'][version] = new_version
versions_info['latest'] = version
with open('update/versions.json', 'w') as jsonf:
json.dump(versions_info, jsonf, indent=4, sort_keys=True)
with open('update/versions.json', 'w', encoding='utf-8') as jsonf:
json.dumps(versions_info, jsonf, indent=4, sort_keys=True)

View File

@ -1,31 +1,29 @@
#!/usr/bin/env python3
from __future__ import unicode_literals
import hashlib
import urllib.request
import json
import os.path
import sys
versions_info = json.load(open('update/versions.json'))
dirn = os.path.dirname
sys.path.insert(0, dirn(dirn((os.path.abspath(__file__)))))
from utils import read_file, write_file
versions_info = json.loads(read_file('update/versions.json'))
version = versions_info['latest']
URL = versions_info['versions'][version]['bin'][0]
data = urllib.request.urlopen(URL).read()
version_dict = versions_info['versions'][version]
# Read template page
with open('download.html.in', 'r', encoding='utf-8') as tmplf:
template = tmplf.read()
template = read_file('download.html.in')
md5sum = hashlib.md5(data).hexdigest()
sha1sum = hashlib.sha1(data).hexdigest()
sha256sum = hashlib.sha256(data).hexdigest()
template = template.replace('@PROGRAM_VERSION@', version)
template = template.replace('@PROGRAM_URL@', URL)
template = template.replace('@PROGRAM_MD5SUM@', md5sum)
template = template.replace('@PROGRAM_SHA1SUM@', sha1sum)
template = template.replace('@PROGRAM_SHA256SUM@', sha256sum)
template = template.replace('@EXE_URL@', versions_info['versions'][version]['exe'][0])
template = template.replace('@EXE_SHA256SUM@', versions_info['versions'][version]['exe'][1])
template = template.replace('@TAR_URL@', versions_info['versions'][version]['tar'][0])
template = template.replace('@TAR_SHA256SUM@', versions_info['versions'][version]['tar'][1])
with open('download.html', 'w', encoding='utf-8') as dlf:
dlf.write(template)
template = template.replace('@PROGRAM_URL@', version_dict['bin'][0])
template = template.replace('@PROGRAM_SHA256SUM@', version_dict['bin'][1])
template = template.replace('@EXE_URL@', version_dict['exe'][0])
template = template.replace('@EXE_SHA256SUM@', version_dict['exe'][1])
template = template.replace('@TAR_URL@', version_dict['tar'][0])
template = template.replace('@TAR_SHA256SUM@', version_dict['tar'][1])
write_file('download.html', template)

View File

@ -5,17 +5,22 @@ from __future__ import with_statement, unicode_literals
import datetime
import glob
import io # For Python 2 compatibility
import os
import re
import sys
year = str(datetime.datetime.now().year)
dirn = os.path.dirname
sys.path.insert(0, dirn(dirn(dirn(os.path.abspath(__file__)))))
from devscripts.utils import read_file, write_file
from youtube_dl import compat_str
year = compat_str(datetime.datetime.now().year)
for fn in glob.glob('*.html*'):
with io.open(fn, encoding='utf-8') as f:
content = f.read()
newc = re.sub(r'(?P<copyright>Copyright © 2006-)(?P<year>[0-9]{4})', 'Copyright © 2006-' + year, content)
content = read_file(fn)
newc = re.sub(r'(?P<copyright>Copyright © 2011-)(?P<year>[0-9]{4})', 'Copyright © 2011-' + year, content)
if content != newc:
tmpFn = fn + '.part'
with io.open(tmpFn, 'wt', encoding='utf-8') as outf:
outf.write(newc)
write_file(tmpFn, newc)
os.rename(tmpFn, fn)

View File

@ -2,15 +2,21 @@
from __future__ import unicode_literals
import datetime
import io
import json
import os.path
import textwrap
import sys
dirn = os.path.dirname
sys.path.insert(0, dirn(dirn(os.path.abspath(__file__))))
from utils import write_file
atom_template = textwrap.dedent("""\
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<link rel="self" href="http://rg3.github.io/youtube-dl/update/releases.atom" />
<link rel="self" href="http://ytdl-org.github.io/youtube-dl/update/releases.atom" />
<title>youtube-dl releases</title>
<id>https://yt-dl.org/feed/youtube-dl-updates-feed</id>
<updated>@TIMESTAMP@</updated>
@ -21,7 +27,7 @@ entry_template = textwrap.dedent("""
<entry>
<id>https://yt-dl.org/feed/youtube-dl-updates-feed/youtube-dl-@VERSION@</id>
<title>New version @VERSION@</title>
<link href="http://rg3.github.io/youtube-dl" />
<link href="http://ytdl-org.github.io/youtube-dl" />
<content type="xhtml">
<div xmlns="http://www.w3.org/1999/xhtml">
Downloads available at <a href="https://yt-dl.org/downloads/@VERSION@/">https://yt-dl.org/downloads/@VERSION@/</a>
@ -72,5 +78,4 @@ for v in versions:
entries_str = textwrap.indent(''.join(entries), '\t')
atom_template = atom_template.replace('@ENTRIES@', entries_str)
with io.open('update/releases.atom', 'w', encoding='utf-8') as atom_file:
atom_file.write(atom_template)
write_file('update/releases.atom', atom_template)

View File

@ -5,15 +5,17 @@ import sys
import os
import textwrap
dirn = os.path.dirname
# We must be able to import youtube_dl
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
sys.path.insert(0, dirn(dirn(dirn(os.path.abspath(__file__)))))
import youtube_dl
from devscripts.utils import read_file, write_file
def main():
with open('supportedsites.html.in', 'r', encoding='utf-8') as tmplf:
template = tmplf.read()
template = read_file('supportedsites.html.in')
ie_htmls = []
for ie in youtube_dl.list_extractors(age_limit=None):
@ -29,8 +31,8 @@ def main():
template = template.replace('@SITES@', textwrap.indent('\n'.join(ie_htmls), '\t'))
with open('supportedsites.html', 'w', encoding='utf-8') as sitesf:
sitesf.write(template)
write_file('supportedsites.html', template)
if __name__ == '__main__':
main()

View File

@ -0,0 +1,19 @@
# coding: utf-8
from __future__ import unicode_literals
import re
class LazyLoadExtractor(object):
_module = None
@classmethod
def ie_key(cls):
return cls.__name__[:-2]
def __new__(cls, *args, **kwargs):
mod = __import__(cls._module, fromlist=(cls.__name__,))
real_cls = getattr(mod, cls.__name__)
instance = real_cls.__new__(real_cls)
instance.__init__(*args, **kwargs)
return instance

View File

@ -1,10 +1,11 @@
#!/usr/bin/env python
from __future__ import unicode_literals
import io
import optparse
import re
from utils import read_file, write_file
def main():
parser = optparse.OptionParser(usage='%prog INFILE OUTFILE')
@ -14,8 +15,7 @@ def main():
infile, outfile = args
with io.open(infile, encoding='utf-8') as inf:
readme = inf.read()
readme = read_file(infile)
bug_text = re.search(
r'(?s)#\s*BUGS\s*[^\n]*\s*(.*?)#\s*COPYRIGHT', readme).group(1)
@ -25,8 +25,8 @@ def main():
out = bug_text + dev_text
with io.open(outfile, 'w', encoding='utf-8') as outf:
outf.write(out)
write_file(outfile, out)
if __name__ == '__main__':
main()

View File

@ -0,0 +1,26 @@
#!/usr/bin/env python
from __future__ import unicode_literals
import optparse
import os.path
import sys
from utils import read_file, read_version, write_file
def main():
parser = optparse.OptionParser(usage='%prog INFILE OUTFILE')
options, args = parser.parse_args()
if len(args) != 2:
parser.error('Expected an input and an output filename')
infile, outfile = args
issue_template_tmpl = read_file(infile)
out = issue_template_tmpl % {'version': read_version()}
write_file(outfile, out)
if __name__ == '__main__':
main()

View File

@ -0,0 +1,131 @@
from __future__ import unicode_literals, print_function
from inspect import getsource
import os
from os.path import dirname as dirn
import re
import sys
print('WARNING: Lazy loading extractors is an experimental feature that may not always work', file=sys.stderr)
sys.path.insert(0, dirn(dirn(os.path.abspath(__file__))))
lazy_extractors_filename = sys.argv[1]
if os.path.exists(lazy_extractors_filename):
os.remove(lazy_extractors_filename)
# Py2: may be confused by leftover lazy_extractors.pyc
if sys.version_info[0] < 3:
for c in ('c', 'o'):
try:
os.remove(lazy_extractors_filename + 'c')
except OSError:
pass
from devscripts.utils import read_file, write_file
from youtube_dl.compat import compat_register_utf8
compat_register_utf8()
from youtube_dl.extractor import _ALL_CLASSES
from youtube_dl.extractor.common import InfoExtractor, SearchInfoExtractor
module_template = read_file('devscripts/lazy_load_template.py')
def get_source(m):
return re.sub(r'(?m)^\s*#.*\n', '', getsource(m))
module_contents = [
module_template,
get_source(InfoExtractor.suitable),
get_source(InfoExtractor._match_valid_url) + '\n',
'class LazyLoadSearchExtractor(LazyLoadExtractor):\n pass\n',
# needed for suitable() methods of Youtube extractor (see #28780)
'from youtube_dl.utils import parse_qs, variadic\n',
]
ie_template = '''
class {name}({bases}):
_VALID_URL = {valid_url!r}
_module = '{module}'
'''
make_valid_template = '''
@classmethod
def _make_valid_url(cls):
return {valid_url!r}
'''
def get_base_name(base):
if base is InfoExtractor:
return 'LazyLoadExtractor'
elif base is SearchInfoExtractor:
return 'LazyLoadSearchExtractor'
else:
return base.__name__
def build_lazy_ie(ie, name):
valid_url = getattr(ie, '_VALID_URL', None)
s = ie_template.format(
name=name,
bases=', '.join(map(get_base_name, ie.__bases__)),
valid_url=valid_url,
module=ie.__module__)
if ie.suitable.__func__ is not InfoExtractor.suitable.__func__:
s += '\n' + get_source(ie.suitable)
if hasattr(ie, '_make_valid_url'):
# search extractors
s += make_valid_template.format(valid_url=ie._make_valid_url())
return s
# find the correct sorting and add the required base classes so that subclasses
# can be correctly created
classes = _ALL_CLASSES[:-1]
ordered_cls = []
while classes:
for c in classes[:]:
bases = set(c.__bases__) - set((object, InfoExtractor, SearchInfoExtractor))
stop = False
for b in bases:
if b not in classes and b not in ordered_cls:
if b.__name__ == 'GenericIE':
exit()
classes.insert(0, b)
stop = True
if stop:
break
if all(b in ordered_cls for b in bases):
ordered_cls.append(c)
classes.remove(c)
break
ordered_cls.append(_ALL_CLASSES[-1])
names = []
for ie in ordered_cls:
name = ie.__name__
src = build_lazy_ie(ie, name)
module_contents.append(src)
if ie in _ALL_CLASSES:
names.append(name)
module_contents.append(
'_ALL_CLASSES = [{0}]'.format(', '.join(names)))
module_src = '\n'.join(module_contents)
write_file(lazy_extractors_filename, module_src + '\n')
# work around JVM byte code module limit in Jython
if sys.platform.startswith('java') and sys.version_info[:2] == (2, 7):
import subprocess
from youtube_dl.compat import compat_subprocess_get_DEVNULL
# if Python 2.7 is available, use it to compile the module for Jython
try:
# if Python 2.7 is available, use it to compile the module for Jython
subprocess.check_call(['python2.7', '-m', 'py_compile', lazy_extractors_filename], stdout=compat_subprocess_get_DEVNULL())
except Exception:
pass

View File

@ -1,8 +1,14 @@
from __future__ import unicode_literals
import io
import sys
import os.path
import re
import sys
dirn = os.path.dirname
sys.path.insert(0, dirn(dirn(os.path.abspath(__file__))))
from utils import read_file
from youtube_dl.compat import compat_open as open
README_FILE = 'README.md'
helptext = sys.stdin.read()
@ -10,8 +16,7 @@ helptext = sys.stdin.read()
if isinstance(helptext, bytes):
helptext = helptext.decode('utf-8')
with io.open(README_FILE, encoding='utf-8') as f:
oldreadme = f.read()
oldreadme = read_file(README_FILE)
header = oldreadme[:oldreadme.index('# OPTIONS')]
footer = oldreadme[oldreadme.index('# CONFIGURATION'):]
@ -20,7 +25,7 @@ options = helptext[helptext.index(' General Options:') + 19:]
options = re.sub(r'(?m)^ (\w.+)$', r'## \1', options)
options = '# OPTIONS\n' + options + '\n'
with io.open(README_FILE, 'w', encoding='utf-8') as f:
with open(README_FILE, 'w', encoding='utf-8') as f:
f.write(header)
f.write(options)
f.write(footer)

View File

@ -1,17 +1,19 @@
#!/usr/bin/env python
from __future__ import unicode_literals
import io
import optparse
import os
import os.path
import sys
# Import youtube_dl
ROOT_DIR = os.path.join(os.path.dirname(__file__), '..')
sys.path.insert(0, ROOT_DIR)
dirn = os.path.dirname
sys.path.insert(0, dirn(dirn(os.path.abspath(__file__))))
import youtube_dl
from utils import write_file
def main():
parser = optparse.OptionParser(usage='%prog OUTFILE.md')
@ -38,8 +40,8 @@ def main():
' - ' + md + '\n'
for md in gen_ies_md(ies))
with io.open(outfile, 'w', encoding='utf-8') as outf:
outf.write(out)
write_file(outfile, out)
if __name__ == '__main__':
main()

View File

@ -1,12 +1,43 @@
from __future__ import unicode_literals
import io
import optparse
import os.path
import sys
import re
from utils import read_file, write_file
ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
README_FILE = os.path.join(ROOT_DIR, 'README.md')
PREFIX = r'''%YOUTUBE-DL(1)
# NAME
youtube\-dl \- download videos from youtube.com or other video platforms
# SYNOPSIS
**youtube-dl** \[OPTIONS\] URL [URL...]
'''
def main():
parser = optparse.OptionParser(usage='%prog OUTFILE.md')
options, args = parser.parse_args()
if len(args) != 1:
parser.error('Expected an output filename')
outfile, = args
readme = read_file(README_FILE)
readme = re.sub(r'(?s)^.*?(?=# DESCRIPTION)', '', readme)
readme = re.sub(r'\s+youtube-dl \[OPTIONS\] URL \[URL\.\.\.\]', '', readme)
readme = PREFIX + readme
readme = filter_options(readme)
write_file(outfile, readme)
def filter_options(readme):
@ -21,43 +52,26 @@ def filter_options(readme):
if in_options:
if line.lstrip().startswith('-'):
option, description = re.split(r'\s{2,}', line.lstrip())
split_option = option.split(' ')
split = re.split(r'\s{2,}', line.lstrip())
# Description string may start with `-` as well. If there is
# only one piece then it's a description bit not an option.
if len(split) > 1:
option, description = split
split_option = option.split(' ')
if not split_option[-1].startswith('-'): # metavar
option = ' '.join(split_option[:-1] + ['*%s*' % split_option[-1]])
if not split_option[-1].startswith('-'): # metavar
option = ' '.join(split_option[:-1] + ['*%s*' % split_option[-1]])
# Pandoc's definition_lists. See http://pandoc.org/README.html
# for more information.
ret += '\n%s\n: %s\n' % (option, description)
else:
ret += line.lstrip() + '\n'
# Pandoc's definition_lists. See http://pandoc.org/README.html
# for more information.
ret += '\n%s\n: %s\n' % (option, description)
continue
ret += line.lstrip() + '\n'
else:
ret += line + '\n'
return ret
with io.open(README_FILE, encoding='utf-8') as f:
readme = f.read()
PREFIX = '''%YOUTUBE-DL(1)
# NAME
youtube\-dl \- download videos from youtube.com or other video platforms
# SYNOPSIS
**youtube-dl** \[OPTIONS\] URL [URL...]
'''
readme = re.sub(r'(?s)^.*?(?=# DESCRIPTION)', '', readme)
readme = re.sub(r'\s+youtube-dl \[OPTIONS\] URL \[URL\.\.\.\]', '', readme)
readme = PREFIX + readme
readme = filter_options(readme)
if sys.version_info < (3, 0):
print(readme.encode('utf-8'))
else:
print(readme)
if __name__ == '__main__':
main()

View File

@ -6,7 +6,7 @@
# * the git config user.signingkey is properly set
# You will need
# pip install coverage nose rsa
# pip install coverage nose rsa wheel
# TODO
# release notes
@ -15,10 +15,33 @@
set -e
skip_tests=true
if [ "$1" = '--run-tests' ]; then
skip_tests=false
shift
fi
gpg_sign_commits=""
buildserver='localhost:8142'
while true
do
case "$1" in
--run-tests)
skip_tests=false
shift
;;
--gpg-sign-commits|-S)
gpg_sign_commits="-S"
shift
;;
--buildserver)
buildserver="$2"
shift 2
;;
--*)
echo "ERROR: unknown option $1"
exit 1
;;
*)
break
;;
esac
done
if [ -z "$1" ]; then echo "ERROR: specify version number like this: $0 1994.09.06"; exit 1; fi
version="$1"
@ -33,6 +56,12 @@ if [ ! -z "`git status --porcelain | grep -v CHANGELOG`" ]; then echo 'ERROR: th
useless_files=$(find youtube_dl -type f -not -name '*.py')
if [ ! -z "$useless_files" ]; then echo "ERROR: Non-.py files in youtube_dl: $useless_files"; exit 1; fi
if [ ! -f "updates_key.pem" ]; then echo 'ERROR: updates_key.pem missing'; exit 1; fi
if ! type pandoc >/dev/null 2>/dev/null; then echo 'ERROR: pandoc is missing'; exit 1; fi
if ! python3 -c 'import rsa' 2>/dev/null; then echo 'ERROR: python3-rsa is missing'; exit 1; fi
if ! python3 -c 'import wheel' 2>/dev/null; then echo 'ERROR: wheel is missing'; exit 1; fi
read -p "Is ChangeLog up to date? (y/n) " -n 1
if [[ ! $REPLY =~ ^[Yy]$ ]]; then exit 1; fi
/bin/echo -e "\n### First of all, testing..."
make clean
@ -45,10 +74,13 @@ fi
/bin/echo -e "\n### Changing version in version.py..."
sed -i "s/__version__ = '.*'/__version__ = '$version'/" youtube_dl/version.py
/bin/echo -e "\n### Committing documentation and youtube_dl/version.py..."
make README.md CONTRIBUTING.md supportedsites
git add README.md CONTRIBUTING.md docs/supportedsites.md youtube_dl/version.py
git commit -m "release $version"
/bin/echo -e "\n### Changing version in ChangeLog..."
sed -i "s/<unreleased>/$version/" ChangeLog
/bin/echo -e "\n### Committing documentation, templates and youtube_dl/version.py..."
make README.md CONTRIBUTING.md issuetemplates supportedsites
git add README.md CONTRIBUTING.md .github/ISSUE_TEMPLATE/1_broken_site.md .github/ISSUE_TEMPLATE/2_site_support_request.md .github/ISSUE_TEMPLATE/3_site_feature_request.md .github/ISSUE_TEMPLATE/4_bug_report.md .github/ISSUE_TEMPLATE/5_feature_request.md .github/ISSUE_TEMPLATE/6_question.md docs/supportedsites.md youtube_dl/version.py ChangeLog
git commit $gpg_sign_commits -m "release $version"
/bin/echo -e "\n### Now tagging, signing and pushing..."
git tag -s -m "Release $version" "$version"
@ -64,7 +96,7 @@ git push origin "$version"
REV=$(git rev-parse HEAD)
make youtube-dl youtube-dl.tar.gz
read -p "VM running? (y/n) " -n 1
wget "http://localhost:8142/build/rg3/youtube-dl/youtube-dl.exe?rev=$REV" -O youtube-dl.exe
wget "http://$buildserver/build/ytdl-org/youtube-dl/youtube-dl.exe?rev=$REV" -O youtube-dl.exe
mkdir -p "build/$version"
mv youtube-dl youtube-dl.exe "build/$version"
mv youtube-dl.tar.gz "build/$version/youtube-dl-$version.tar.gz"
@ -74,15 +106,16 @@ RELEASE_FILES="youtube-dl youtube-dl.exe youtube-dl-$version.tar.gz"
(cd build/$version/ && sha256sum $RELEASE_FILES > SHA2-256SUMS)
(cd build/$version/ && sha512sum $RELEASE_FILES > SHA2-512SUMS)
/bin/echo -e "\n### Signing and uploading the new binaries to yt-dl.org ..."
/bin/echo -e "\n### Signing and uploading the new binaries to GitHub..."
for f in $RELEASE_FILES; do gpg --passphrase-repeat 5 --detach-sig "build/$version/$f"; done
scp -r "build/$version" ytdl@yt-dl.org:html/tmp/
ssh ytdl@yt-dl.org "mv html/tmp/$version html/downloads/"
ROOT=$(pwd)
python devscripts/create-github-release.py ChangeLog $version "$ROOT/build/$version"
ssh ytdl@yt-dl.org "sh html/update_latest.sh $version"
/bin/echo -e "\n### Now switching to gh-pages..."
git clone --branch gh-pages --single-branch . build/gh-pages
ROOT=$(pwd)
(
set -e
ORIGIN_URL=$(git config --get remote.origin.url)
@ -94,7 +127,7 @@ ROOT=$(pwd)
"$ROOT/devscripts/gh-pages/update-copyright.py"
"$ROOT/devscripts/gh-pages/update-sites.py"
git add *.html *.html.in update
git commit -m "release $version"
git commit $gpg_sign_commits -m "release $version"
git push "$ROOT" gh-pages
git push "$ORIGIN_URL" gh-pages
)

17
devscripts/run_tests.bat Normal file
View File

@ -0,0 +1,17 @@
@echo off
rem Keep this list in sync with the `offlinetest` target in Makefile
set DOWNLOAD_TESTS="age_restriction^|download^|iqiyi_sdk_interpreter^|socks^|subtitles^|write_annotations^|youtube_lists^|youtube_signature"
if "%YTDL_TEST_SET%" == "core" (
set test_set="-I test_("%DOWNLOAD_TESTS%")\.py"
set multiprocess_args=""
) else if "%YTDL_TEST_SET%" == "download" (
set test_set="-I test_(?!"%DOWNLOAD_TESTS%").+\.py"
set multiprocess_args="--processes=4 --process-timeout=540"
) else (
echo YTDL_TEST_SET is not set or invalid
exit /b 1
)
nosetests test --verbose %test_set:"=% %multiprocess_args:"=%

22
devscripts/run_tests.sh Executable file
View File

@ -0,0 +1,22 @@
#!/bin/bash
# Keep this list in sync with the `offlinetest` target in Makefile
DOWNLOAD_TESTS="age_restriction|download|iqiyi_sdk_interpreter|socks|subtitles|write_annotations|youtube_lists|youtube_signature"
test_set=""
multiprocess_args=""
case "$YTDL_TEST_SET" in
core)
test_set="-I test_($DOWNLOAD_TESTS)\.py"
;;
download)
test_set="-I test_(?!$DOWNLOAD_TESTS).+\.py"
multiprocess_args="--processes=4 --process-timeout=540"
;;
*)
break
;;
esac
nosetests test --verbose $test_set $multiprocess_args

View File

@ -0,0 +1,47 @@
#!/usr/bin/env python
from __future__ import unicode_literals
import itertools
import json
import os
import re
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from youtube_dl.compat import (
compat_print,
compat_urllib_request,
)
from youtube_dl.utils import format_bytes
def format_size(bytes):
return '%s (%d bytes)' % (format_bytes(bytes), bytes)
total_bytes = 0
for page in itertools.count(1):
releases = json.loads(compat_urllib_request.urlopen(
'https://api.github.com/repos/ytdl-org/youtube-dl/releases?page=%s' % page
).read().decode('utf-8'))
if not releases:
break
for release in releases:
compat_print(release['name'])
for asset in release['assets']:
asset_name = asset['name']
total_bytes += asset['download_count'] * asset['size']
if all(not re.match(p, asset_name) for p in (
r'^youtube-dl$',
r'^youtube-dl-\d{4}\.\d{2}\.\d{2}(?:\.\d+)?\.tar\.gz$',
r'^youtube-dl\.exe$')):
continue
compat_print(
' %s size: %s downloads: %d'
% (asset_name, format_size(asset['size']), asset['download_count']))
compat_print('total downloads traffic: %s' % format_size(total_bytes))

62
devscripts/utils.py Normal file
View File

@ -0,0 +1,62 @@
# coding: utf-8
from __future__ import unicode_literals
import argparse
import functools
import os.path
import subprocess
import sys
dirn = os.path.dirname
sys.path.insert(0, dirn(dirn(os.path.abspath(__file__))))
from youtube_dl.compat import (
compat_kwargs,
compat_open as open,
)
def read_file(fname):
with open(fname, encoding='utf-8') as f:
return f.read()
def write_file(fname, content, mode='w'):
with open(fname, mode, encoding='utf-8') as f:
return f.write(content)
def read_version(fname='youtube_dl/version.py'):
"""Get the version without importing the package"""
exec(compile(read_file(fname), fname, 'exec'))
return locals()['__version__']
def get_filename_args(has_infile=False, default_outfile=None):
parser = argparse.ArgumentParser()
if has_infile:
parser.add_argument('infile', help='Input file')
kwargs = {'nargs': '?', 'default': default_outfile} if default_outfile else {}
kwargs['help'] = 'Output file'
parser.add_argument('outfile', **compat_kwargs(kwargs))
opts = parser.parse_args()
if has_infile:
return opts.infile, opts.outfile
return opts.outfile
def compose_functions(*functions):
return lambda x: functools.reduce(lambda y, f: f(y), functions, x)
def run_process(*args, **kwargs):
kwargs.setdefault('text', True)
kwargs.setdefault('check', True)
kwargs.setdefault('capture_output', True)
if kwargs['text']:
kwargs.setdefault('encoding', 'utf-8')
kwargs.setdefault('errors', 'replace')
kwargs = compat_kwargs(kwargs)
return subprocess.run(args, **kwargs)

View File

@ -7,6 +7,8 @@ import sys
sys.path.insert(0, dirn(dirn((os.path.abspath(__file__)))))
import youtube_dl
from utils import read_file, write_file
ZSH_COMPLETION_FILE = "youtube-dl.zsh"
ZSH_COMPLETION_TEMPLATE = "devscripts/zsh-completion.in"
@ -34,15 +36,14 @@ def build_completion(opt_parser):
flags = [opt.get_opt_string() for opt in opts]
with open(ZSH_COMPLETION_TEMPLATE) as f:
template = f.read()
template = read_file(ZSH_COMPLETION_TEMPLATE)
template = template.replace("{{fileopts}}", "|".join(fileopts))
template = template.replace("{{diropts}}", "|".join(diropts))
template = template.replace("{{flags}}", " ".join(flags))
with open(ZSH_COMPLETION_FILE, "w") as f:
f.write(template)
write_file(ZSH_COMPLETION_FILE, template)
parser = youtube_dl.parseOpts()[0]
build_completion(parser)

View File

@ -1,4 +1,4 @@
# -*- coding: utf-8 -*-
# coding: utf-8
#
# youtube-dl documentation build configuration file, created by
# sphinx-quickstart on Fri Mar 14 21:05:43 2014.

File diff suppressed because it is too large Load Diff

View File

@ -2,5 +2,5 @@
universal = True
[flake8]
exclude = youtube_dl/extractor/__init__.py,devscripts/buildserver.py,setup.py,build,.git
ignore = E402,E501,E731
exclude = youtube_dl/extractor/__init__.py,devscripts/buildserver.py,devscripts/lazy_load_template.py,devscripts/make_issue_template.py,setup.py,build,.git,venv
ignore = E402,E501,E731,E741,W503

View File

@ -1,5 +1,5 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# coding: utf-8
from __future__ import print_function
@ -8,11 +8,12 @@ import warnings
import sys
try:
from setuptools import setup
from setuptools import setup, Command
setuptools_available = True
except ImportError:
from distutils.core import setup
from distutils.core import setup, Command
setuptools_available = False
from distutils.spawn import spawn
try:
# This will create an exe that needs Microsoft Visual C++ 2008
@ -20,25 +21,37 @@ try:
import py2exe
except ImportError:
if len(sys.argv) >= 2 and sys.argv[1] == 'py2exe':
print("Cannot import py2exe", file=sys.stderr)
print('Cannot import py2exe', file=sys.stderr)
exit(1)
py2exe_options = {
"bundle_files": 1,
"compressed": 1,
"optimize": 2,
"dist_dir": '.',
"dll_excludes": ['w9xpopen.exe', 'crypt32.dll'],
'bundle_files': 1,
'compressed': 1,
'optimize': 2,
'dist_dir': '.',
'dll_excludes': ['w9xpopen.exe', 'crypt32.dll'],
}
# Get the version from youtube_dl/version.py without importing the package
exec(compile(open('youtube_dl/version.py').read(),
'youtube_dl/version.py', 'exec'))
DESCRIPTION = 'YouTube video downloader'
LONG_DESCRIPTION = 'Command-line program to download videos from YouTube.com and other video sites'
py2exe_console = [{
"script": "./youtube_dl/__main__.py",
"dest_base": "youtube-dl",
'script': './youtube_dl/__main__.py',
'dest_base': 'youtube-dl',
'version': __version__,
'description': DESCRIPTION,
'comments': LONG_DESCRIPTION,
'product_name': 'youtube-dl',
'product_version': __version__,
}]
py2exe_params = {
'console': py2exe_console,
'options': {"py2exe": py2exe_options},
'options': {'py2exe': py2exe_options},
'zipfile': None
}
@ -70,21 +83,33 @@ else:
else:
params['scripts'] = ['bin/youtube-dl']
# Get the version from youtube_dl/version.py without importing the package
exec(compile(open('youtube_dl/version.py').read(),
'youtube_dl/version.py', 'exec'))
class build_lazy_extractors(Command):
description = 'Build the extractor lazy loading module'
user_options = []
def initialize_options(self):
pass
def finalize_options(self):
pass
def run(self):
spawn(
[sys.executable, 'devscripts/make_lazy_extractors.py', 'youtube_dl/extractor/lazy_extractors.py'],
dry_run=self.dry_run,
)
setup(
name='youtube_dl',
version=__version__,
description='YouTube video downloader',
long_description='Small command-line program to download videos from'
' YouTube.com and other video sites.',
url='https://github.com/rg3/youtube-dl',
description=DESCRIPTION,
long_description=LONG_DESCRIPTION,
url='https://github.com/ytdl-org/youtube-dl',
author='Ricardo Garcia',
author_email='ytdl@yt-dl.org',
maintainer='Philipp Hagemeister',
maintainer_email='phihag@phihag.de',
maintainer='Sergey M.',
maintainer_email='dstftw@gmail.com',
license='Unlicense',
packages=[
'youtube_dl',
'youtube_dl.extractor', 'youtube_dl.downloader',
@ -95,17 +120,29 @@ setup(
# test_requires = ['nosetest'],
classifiers=[
"Topic :: Multimedia :: Video",
"Development Status :: 5 - Production/Stable",
"Environment :: Console",
"License :: Public Domain",
"Programming Language :: Python :: 2.6",
"Programming Language :: Python :: 2.7",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.2",
"Programming Language :: Python :: 3.3",
"Programming Language :: Python :: 3.4",
'Topic :: Multimedia :: Video',
'Development Status :: 5 - Production/Stable',
'Environment :: Console',
'License :: Public Domain',
'Programming Language :: Python',
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.6',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.2',
'Programming Language :: Python :: 3.3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: Implementation',
'Programming Language :: Python :: Implementation :: CPython',
'Programming Language :: Python :: Implementation :: IronPython',
'Programming Language :: Python :: Implementation :: Jython',
'Programming Language :: Python :: Implementation :: PyPy',
],
cmdclass={'build_lazy_extractors': build_lazy_extractors},
**params
)

View File

@ -1,18 +1,24 @@
from __future__ import unicode_literals
import errno
import io
import hashlib
import json
import os.path
import re
import types
import ssl
import sys
import types
import unittest
import youtube_dl.extractor
from youtube_dl import YoutubeDL
from youtube_dl.utils import (
from youtube_dl.compat import (
compat_open as open,
compat_os_name,
compat_str,
)
from youtube_dl.utils import (
IDENTITY,
preferredencoding,
write_string,
)
@ -21,8 +27,13 @@ from youtube_dl.utils import (
def get_params(override=None):
PARAMETERS_FILE = os.path.join(os.path.dirname(os.path.abspath(__file__)),
"parameters.json")
with io.open(PARAMETERS_FILE, encoding='utf-8') as pf:
LOCAL_PARAMETERS_FILE = os.path.join(os.path.dirname(os.path.abspath(__file__)),
"local_parameters.json")
with open(PARAMETERS_FILE, encoding='utf-8') as pf:
parameters = json.load(pf)
if os.path.exists(LOCAL_PARAMETERS_FILE):
with open(LOCAL_PARAMETERS_FILE, encoding='utf-8') as pf:
parameters.update(json.load(pf))
if override:
parameters.update(override)
return parameters
@ -42,7 +53,7 @@ def report_warning(message):
Print the message to stderr, it will be prefixed with 'WARNING:'
If stderr is a tty file the 'WARNING:' will be colored
'''
if sys.stderr.isatty() and os.name != 'nt':
if sys.stderr.isatty() and compat_os_name != 'nt':
_msg_header = '\033[0;33mWARNING:\033[0m'
else:
_msg_header = 'WARNING:'
@ -63,7 +74,8 @@ class FakeYDL(YoutubeDL):
def to_screen(self, s, skip_eol=None):
print(s)
def trouble(self, s, tb=None):
def trouble(self, *args, **kwargs):
s = args[0] if len(args) > 0 else kwargs.get('message', 'Missing message')
raise Exception(s)
def download(self, x):
@ -80,6 +92,17 @@ class FakeYDL(YoutubeDL):
self.report_warning = types.MethodType(report_warning, self)
class FakeLogger(object):
def debug(self, msg):
pass
def warning(self, msg):
pass
def error(self, msg):
pass
def gettestcases(include_onlymatching=False):
for ie in youtube_dl.extractor.gen_extractors():
for tc in ie.get_testcases(include_onlymatching):
@ -119,6 +142,12 @@ def expect_value(self, got, expected, field):
self.assertTrue(
contains_str in got,
'field %s (value: %r) should contain %r' % (field, got, contains_str))
elif isinstance(expected, compat_str) and re.match(r'lambda \w+:', expected):
fn = eval(expected)
suite = expected.split(':', 1)[1].strip()
self.assertTrue(
fn(got),
'Expected field %s to meet condition %s, but value %r failed ' % (field, suite, got))
elif isinstance(expected, type):
self.assertTrue(
isinstance(got, expected),
@ -128,7 +157,7 @@ def expect_value(self, got, expected, field):
elif isinstance(expected, list) and isinstance(got, list):
self.assertEqual(
len(expected), len(got),
'Expect a list of length %d, but got a list of length %d for field %s' % (
'Expected a list of length %d, but got a list of length %d for field %s' % (
len(expected), len(got), field))
for index, (item_got, item_expected) in enumerate(zip(got, expected)):
type_got = type(item_got)
@ -140,16 +169,31 @@ def expect_value(self, got, expected, field):
expect_value(self, item_got, item_expected, field)
else:
if isinstance(expected, compat_str) and expected.startswith('md5:'):
self.assertTrue(
isinstance(got, compat_str),
'Expected field %s to be a unicode object, but got value %r of type %r' % (field, got, type(got)))
got = 'md5:' + md5(got)
elif isinstance(expected, compat_str) and expected.startswith('mincount:'):
elif isinstance(expected, compat_str) and re.match(r'^(?:min|max)?count:\d+', expected):
self.assertTrue(
isinstance(got, (list, dict)),
'Expected field %s to be a list or a dict, but it is of type %s' % (
field, type(got).__name__))
expected_num = int(expected.partition(':')[2])
assertGreaterEqual(
self, len(got), expected_num,
'Expected %d items in field %s, but only got %d' % (expected_num, field, len(got)))
op, _, expected_num = expected.partition(':')
expected_num = int(expected_num)
if op == 'mincount':
assert_func = self.assertGreaterEqual
msg_tmpl = 'Expected %d items in field %s, but only got %d'
elif op == 'maxcount':
assert_func = self.assertLessEqual
msg_tmpl = 'Expected maximum %d items in field %s, but got %d'
elif op == 'count':
assert_func = self.assertEqual
msg_tmpl = 'Expected exactly %d items in field %s, but got %d'
else:
assert False
assert_func(
len(got), expected_num,
msg_tmpl % (expected_num, field, len(got)))
return
self.assertEqual(
expected, got,
@ -218,13 +262,6 @@ def assertRegexpMatches(self, text, regexp, msg=None):
self.assertTrue(m, msg)
def assertGreaterEqual(self, got, expected, msg=None):
if not (got >= expected):
if msg is None:
msg = '%r not greater than or equal to %r' % (got, expected)
self.assertTrue(got >= expected, msg)
def expect_warnings(ydl, warnings_re):
real_warning = ydl.report_warning
@ -233,3 +270,16 @@ def expect_warnings(ydl, warnings_re):
real_warning(w)
ydl.report_warning = _report_warning
def http_server_port(httpd):
if os.name == 'java' and isinstance(httpd.socket, ssl.SSLSocket):
# In Jython SSLSocket is not a subclass of socket.socket
sock = httpd.socket.sock
else:
sock = httpd.socket
return sock.getsockname()[1]
def expectedFailureIf(cond):
return unittest.expectedFailure if cond else IDENTITY

View File

@ -18,7 +18,6 @@
"noprogress": false,
"outtmpl": "%(id)s.%(ext)s",
"password": null,
"playlistend": -1,
"playliststart": 1,
"prefer_free_formats": false,
"quiet": false,
@ -37,7 +36,7 @@
"writeinfojson": true,
"writesubtitles": false,
"allsubtitles": false,
"listssubtitles": false,
"listsubtitles": false,
"socket_timeout": 20,
"fixup": "never"
}

File diff suppressed because it is too large Load Diff

View File

@ -1,4 +1,5 @@
#!/usr/bin/env python
# coding: utf-8
from __future__ import unicode_literals
@ -9,14 +10,31 @@ import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import copy
import json
from test.helper import FakeYDL, assertRegexpMatches
from test.helper import (
FakeYDL,
assertRegexpMatches,
try_rm,
)
from youtube_dl import YoutubeDL
from youtube_dl.compat import compat_str, compat_urllib_error
from youtube_dl.compat import (
compat_http_cookiejar_Cookie,
compat_http_cookies_SimpleCookie,
compat_kwargs,
compat_open as open,
compat_str,
compat_urllib_error,
)
from youtube_dl.extractor import YoutubeIE
from youtube_dl.extractor.common import InfoExtractor
from youtube_dl.postprocessor.common import PostProcessor
from youtube_dl.utils import ExtractorError, match_filter_func
from youtube_dl.utils import (
ExtractorError,
match_filter_func,
traverse_obj,
)
TEST_URL = 'http://localhost/sample.mp4'
@ -28,11 +46,14 @@ class YDL(FakeYDL):
self.msgs = []
def process_info(self, info_dict):
self.downloaded_info_dicts.append(info_dict)
self.downloaded_info_dicts.append(info_dict.copy())
def to_screen(self, msg):
self.msgs.append(msg)
def dl(self, *args, **kwargs):
assert False, 'Downloader must not be invoked for test_YoutubeDL'
def _make_result(formats, **kwargs):
res = {
@ -40,8 +61,10 @@ def _make_result(formats, **kwargs):
'id': 'testid',
'title': 'testttitle',
'extractor': 'testex',
'extractor_key': 'TestEx',
'webpage_url': 'http://example.com/watch?v=shenanigans',
}
res.update(**kwargs)
res.update(**compat_kwargs(kwargs))
return res
@ -222,6 +245,11 @@ class TestFormatSelection(unittest.TestCase):
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'dash-video-low')
ydl = YDL({'format': 'bestvideo[format_id^=dash][format_id$=low]'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'dash-video-low')
formats = [
{'format_id': 'vid-vcodec-dot', 'ext': 'mp4', 'preference': 1, 'vcodec': 'avc1.123456', 'acodec': 'none', 'url': TEST_URL},
]
@ -232,9 +260,79 @@ class TestFormatSelection(unittest.TestCase):
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'vid-vcodec-dot')
def test_format_selection_string_ops(self):
formats = [
{'format_id': 'abc-cba', 'ext': 'mp4', 'url': TEST_URL},
{'format_id': 'zxc-cxz', 'ext': 'webm', 'url': TEST_URL},
]
info_dict = _make_result(formats)
# equals (=)
ydl = YDL({'format': '[format_id=abc-cba]'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'abc-cba')
# does not equal (!=)
ydl = YDL({'format': '[format_id!=abc-cba]'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'zxc-cxz')
ydl = YDL({'format': '[format_id!=abc-cba][format_id!=zxc-cxz]'})
self.assertRaises(ExtractorError, ydl.process_ie_result, info_dict.copy())
# starts with (^=)
ydl = YDL({'format': '[format_id^=abc]'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'abc-cba')
# does not start with (!^=)
ydl = YDL({'format': '[format_id!^=abc]'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'zxc-cxz')
ydl = YDL({'format': '[format_id!^=abc][format_id!^=zxc]'})
self.assertRaises(ExtractorError, ydl.process_ie_result, info_dict.copy())
# ends with ($=)
ydl = YDL({'format': '[format_id$=cba]'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'abc-cba')
# does not end with (!$=)
ydl = YDL({'format': '[format_id!$=cba]'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'zxc-cxz')
ydl = YDL({'format': '[format_id!$=cba][format_id!$=cxz]'})
self.assertRaises(ExtractorError, ydl.process_ie_result, info_dict.copy())
# contains (*=)
ydl = YDL({'format': '[format_id*=bc-cb]'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'abc-cba')
# does not contain (!*=)
ydl = YDL({'format': '[format_id!*=bc-cb]'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'zxc-cxz')
ydl = YDL({'format': '[format_id!*=abc][format_id!*=zxc]'})
self.assertRaises(ExtractorError, ydl.process_ie_result, info_dict.copy())
ydl = YDL({'format': '[format_id!*=-]'})
self.assertRaises(ExtractorError, ydl.process_ie_result, info_dict.copy())
def test_youtube_format_selection(self):
order = [
'38', '37', '46', '22', '45', '35', '44', '18', '34', '43', '6', '5', '36', '17', '13',
'38', '37', '46', '22', '45', '35', '44', '18', '34', '43', '6', '5', '17', '36', '13',
# Apple HTTP Live Streaming
'96', '95', '94', '93', '92', '132', '151',
# 3D
@ -330,6 +428,53 @@ class TestFormatSelection(unittest.TestCase):
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], f1['format_id'])
def test_audio_only_extractor_format_selection(self):
# For extractors with incomplete formats (all formats are audio-only or
# video-only) best and worst should fallback to corresponding best/worst
# video-only or audio-only formats (as per
# https://github.com/ytdl-org/youtube-dl/pull/5556)
formats = [
{'format_id': 'low', 'ext': 'mp3', 'preference': 1, 'vcodec': 'none', 'url': TEST_URL},
{'format_id': 'high', 'ext': 'mp3', 'preference': 2, 'vcodec': 'none', 'url': TEST_URL},
]
info_dict = _make_result(formats)
ydl = YDL({'format': 'best'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'high')
ydl = YDL({'format': 'worst'})
ydl.process_ie_result(info_dict.copy())
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'low')
def test_format_not_available(self):
formats = [
{'format_id': 'regular', 'ext': 'mp4', 'height': 360, 'url': TEST_URL},
{'format_id': 'video', 'ext': 'mp4', 'height': 720, 'acodec': 'none', 'url': TEST_URL},
]
info_dict = _make_result(formats)
# This must fail since complete video-audio format does not match filter
# and extractor does not provide incomplete only formats (i.e. only
# video-only or audio-only).
ydl = YDL({'format': 'best[height>360]'})
self.assertRaises(ExtractorError, ydl.process_ie_result, info_dict.copy())
def test_format_selection_issue_10083(self):
# See https://github.com/ytdl-org/youtube-dl/issues/10083
formats = [
{'format_id': 'regular', 'height': 360, 'url': TEST_URL},
{'format_id': 'video', 'height': 720, 'acodec': 'none', 'url': TEST_URL},
{'format_id': 'audio', 'vcodec': 'none', 'url': TEST_URL},
]
info_dict = _make_result(formats)
ydl = YDL({'format': 'best[height>360]/bestvideo[height>360]+bestaudio'})
ydl.process_ie_result(info_dict.copy())
self.assertEqual(ydl.downloaded_info_dicts[0]['format_id'], 'video+audio')
def test_invalid_format_specs(self):
def assert_syntax_error(format_spec):
ydl = YDL({'format': format_spec})
@ -340,6 +485,7 @@ class TestFormatSelection(unittest.TestCase):
assert_syntax_error('+bestaudio')
assert_syntax_error('bestvideo+')
assert_syntax_error('/')
assert_syntax_error('bestvideo+bestvideo+bestaudio')
def test_format_filtering(self):
formats = [
@ -408,6 +554,23 @@ class TestFormatSelection(unittest.TestCase):
pass
self.assertEqual(ydl.downloaded_info_dicts, [])
def test_default_format_spec(self):
ydl = YDL({'simulate': True})
self.assertEqual(ydl._default_format_spec({}), 'bestvideo+bestaudio/best')
ydl = YDL({})
self.assertEqual(ydl._default_format_spec({'is_live': True}), 'best/bestvideo+bestaudio')
ydl = YDL({'simulate': True})
self.assertEqual(ydl._default_format_spec({'is_live': True}), 'bestvideo+bestaudio/best')
ydl = YDL({'outtmpl': '-'})
self.assertEqual(ydl._default_format_spec({}), 'best/bestvideo+bestaudio')
ydl = YDL({})
self.assertEqual(ydl._default_format_spec({}, download=False), 'bestvideo+bestaudio/best')
self.assertEqual(ydl._default_format_spec({'is_live': True}), 'best/bestvideo+bestaudio')
class TestYoutubeDL(unittest.TestCase):
def test_subtitles(self):
@ -486,22 +649,52 @@ class TestYoutubeDL(unittest.TestCase):
'id': '1234',
'ext': 'mp4',
'width': None,
'height': 1080,
'title1': '$PATH',
'title2': '%PATH%',
}
def fname(templ):
ydl = YoutubeDL({'outtmpl': templ})
def fname(templ, na_placeholder='NA'):
params = {'outtmpl': templ}
if na_placeholder != 'NA':
params['outtmpl_na_placeholder'] = na_placeholder
ydl = YoutubeDL(params)
return ydl.prepare_filename(info)
self.assertEqual(fname('%(id)s.%(ext)s'), '1234.mp4')
self.assertEqual(fname('%(id)s-%(width)s.%(ext)s'), '1234-NA.mp4')
# Replace missing fields with 'NA'
self.assertEqual(fname('%(uploader_date)s-%(id)s.%(ext)s'), 'NA-1234.mp4')
NA_TEST_OUTTMPL = '%(uploader_date)s-%(width)d-%(id)s.%(ext)s'
# Replace missing fields with 'NA' by default
self.assertEqual(fname(NA_TEST_OUTTMPL), 'NA-NA-1234.mp4')
# Or by provided placeholder
self.assertEqual(fname(NA_TEST_OUTTMPL, na_placeholder='none'), 'none-none-1234.mp4')
self.assertEqual(fname(NA_TEST_OUTTMPL, na_placeholder=''), '--1234.mp4')
self.assertEqual(fname('%(height)d.%(ext)s'), '1080.mp4')
self.assertEqual(fname('%(height)6d.%(ext)s'), ' 1080.mp4')
self.assertEqual(fname('%(height)-6d.%(ext)s'), '1080 .mp4')
self.assertEqual(fname('%(height)06d.%(ext)s'), '001080.mp4')
self.assertEqual(fname('%(height) 06d.%(ext)s'), ' 01080.mp4')
self.assertEqual(fname('%(height) 06d.%(ext)s'), ' 01080.mp4')
self.assertEqual(fname('%(height)0 6d.%(ext)s'), ' 01080.mp4')
self.assertEqual(fname('%(height)0 6d.%(ext)s'), ' 01080.mp4')
self.assertEqual(fname('%(height) 0 6d.%(ext)s'), ' 01080.mp4')
self.assertEqual(fname('%%'), '%')
self.assertEqual(fname('%%%%'), '%%')
self.assertEqual(fname('%%(height)06d.%(ext)s'), '%(height)06d.mp4')
self.assertEqual(fname('%(width)06d.%(ext)s'), 'NA.mp4')
self.assertEqual(fname('%(width)06d.%%(ext)s'), 'NA.%(ext)s')
self.assertEqual(fname('%%(width)06d.%(ext)s'), '%(width)06d.mp4')
self.assertEqual(fname('Hello %(title1)s'), 'Hello $PATH')
self.assertEqual(fname('Hello %(title2)s'), 'Hello %PATH%')
def test_format_note(self):
ydl = YoutubeDL()
self.assertEqual(ydl._format_note({}), '')
assertRegexpMatches(self, ydl._format_note({
'vbr': 10,
}), '^\s*10k$')
}), r'^\s*10k$')
assertRegexpMatches(self, ydl._format_note({
'fps': 30,
}), r'^30fps$')
def test_postprocessors(self):
filename = 'post-processor-testfile.mp4'
@ -509,12 +702,12 @@ class TestYoutubeDL(unittest.TestCase):
class SimplePP(PostProcessor):
def run(self, info):
with open(audiofile, 'wt') as f:
with open(audiofile, 'w') as f:
f.write('EXAMPLE')
return [info['filepath']], info
def run_pp(params, PP):
with open(filename, 'wt') as f:
with open(filename, 'w') as f:
f.write('EXAMPLE')
ydl = YoutubeDL(params)
ydl.add_post_processor(PP())
@ -533,7 +726,7 @@ class TestYoutubeDL(unittest.TestCase):
class ModifierPP(PostProcessor):
def run(self, info):
with open(info['filepath'], 'wt') as f:
with open(info['filepath'], 'w') as f:
f.write('MODIFIED')
return [], info
@ -563,6 +756,9 @@ class TestYoutubeDL(unittest.TestCase):
'extractor': 'TEST',
'duration': 30,
'filesize': 10 * 1024,
'playlist_id': '42',
'uploader': "變態妍字幕版 太妍 тест",
'creator': "тест ' 123 ' тест--",
}
second = {
'id': '2',
@ -572,6 +768,8 @@ class TestYoutubeDL(unittest.TestCase):
'duration': 10,
'description': 'foo',
'filesize': 5 * 1024,
'playlist_id': '43',
'uploader': "тест 123",
}
videos = [first, second]
@ -608,6 +806,30 @@ class TestYoutubeDL(unittest.TestCase):
res = get_videos(f)
self.assertEqual(res, ['1'])
f = match_filter_func('playlist_id = 42')
res = get_videos(f)
self.assertEqual(res, ['1'])
f = match_filter_func('uploader = "變態妍字幕版 太妍 тест"')
res = get_videos(f)
self.assertEqual(res, ['1'])
f = match_filter_func('uploader != "變態妍字幕版 太妍 тест"')
res = get_videos(f)
self.assertEqual(res, ['2'])
f = match_filter_func('creator = "тест \' 123 \' тест--"')
res = get_videos(f)
self.assertEqual(res, ['1'])
f = match_filter_func("creator = 'тест \\' 123 \\' тест--'")
res = get_videos(f)
self.assertEqual(res, ['1'])
f = match_filter_func(r"creator = 'тест \' 123 \' тест--' & duration > 30")
res = get_videos(f)
self.assertEqual(res, [])
def test_playlist_items_selection(self):
entries = [{
'id': compat_str(i),
@ -623,11 +845,15 @@ class TestYoutubeDL(unittest.TestCase):
'webpage_url': 'http://example.com',
}
def get_ids(params):
def get_downloaded_info_dicts(params):
ydl = YDL(params)
# make a copy because the dictionary can be modified
ydl.process_ie_result(playlist.copy())
return [int(v['id']) for v in ydl.downloaded_info_dicts]
# make a deep copy because the dictionary and nested entries
# can be modified
ydl.process_ie_result(copy.deepcopy(playlist))
return ydl.downloaded_info_dicts
def get_ids(params):
return [int(v['id']) for v in get_downloaded_info_dicts(params)]
result = get_ids({})
self.assertEqual(result, [1, 2, 3, 4])
@ -653,8 +879,30 @@ class TestYoutubeDL(unittest.TestCase):
result = get_ids({'playlist_items': '10'})
self.assertEqual(result, [])
result = get_ids({'playlist_items': '3-10'})
self.assertEqual(result, [3, 4])
result = get_ids({'playlist_items': '2-4,3-4,3'})
self.assertEqual(result, [2, 3, 4])
# Tests for https://github.com/ytdl-org/youtube-dl/issues/10591
# @{
result = get_downloaded_info_dicts({'playlist_items': '2-4,3-4,3'})
self.assertEqual(result[0]['playlist_index'], 2)
self.assertEqual(result[1]['playlist_index'], 3)
result = get_downloaded_info_dicts({'playlist_items': '2-4,3-4,3'})
self.assertEqual(result[0]['playlist_index'], 2)
self.assertEqual(result[1]['playlist_index'], 3)
self.assertEqual(result[2]['playlist_index'], 4)
result = get_downloaded_info_dicts({'playlist_items': '4,2'})
self.assertEqual(result[0]['playlist_index'], 4)
self.assertEqual(result[1]['playlist_index'], 2)
# @}
def test_urlopen_no_file_protocol(self):
# see https://github.com/rg3/youtube-dl/issues/8227
# see https://github.com/ytdl-org/youtube-dl/issues/8227
ydl = YDL()
self.assertRaises(compat_urllib_error.URLError, ydl.urlopen, 'file:///etc/passwd')
@ -669,6 +917,8 @@ class TestYoutubeDL(unittest.TestCase):
'_type': 'url_transparent',
'url': 'foo2:',
'ie_key': 'Foo2',
'title': 'foo1 title',
'id': 'foo1_id',
}
class Foo2IE(InfoExtractor):
@ -685,7 +935,7 @@ class TestYoutubeDL(unittest.TestCase):
_VALID_URL = r'foo3:'
def _real_extract(self, url):
return _make_result([{'url': TEST_URL}])
return _make_result([{'url': TEST_URL}], title='foo3 title')
ydl.add_info_extractor(Foo1IE(ydl))
ydl.add_info_extractor(Foo2IE(ydl))
@ -693,6 +943,248 @@ class TestYoutubeDL(unittest.TestCase):
ydl.extract_info('foo1:')
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['url'], TEST_URL)
self.assertEqual(downloaded['title'], 'foo1 title')
self.assertEqual(downloaded['id'], 'testid')
self.assertEqual(downloaded['extractor'], 'testex')
self.assertEqual(downloaded['extractor_key'], 'TestEx')
# Test case for https://github.com/ytdl-org/youtube-dl/issues/27064
def test_ignoreerrors_for_playlist_with_url_transparent_iterable_entries(self):
ydl = YDL({
'format': 'extra',
'ignoreerrors': True,
})
ydl.trouble = lambda *_, **__: None
class VideoIE(InfoExtractor):
_VALID_URL = r'video:(?P<id>\d+)'
def _real_extract(self, url):
video_id = self._match_id(url)
formats = [{
'format_id': 'default',
'url': 'url:',
}]
if video_id == '0':
raise ExtractorError('foo')
if video_id == '2':
formats.append({
'format_id': 'extra',
'url': TEST_URL,
})
return {
'id': video_id,
'title': 'Video %s' % video_id,
'formats': formats,
}
class PlaylistIE(InfoExtractor):
_VALID_URL = r'playlist:'
def _entries(self):
for n in range(3):
video_id = compat_str(n)
yield {
'_type': 'url_transparent',
'ie_key': VideoIE.ie_key(),
'id': video_id,
'url': 'video:%s' % video_id,
'title': 'Video Transparent %s' % video_id,
}
def _real_extract(self, url):
return self.playlist_result(self._entries())
ydl.add_info_extractor(VideoIE(ydl))
ydl.add_info_extractor(PlaylistIE(ydl))
info = ydl.extract_info('playlist:')
entries = info['entries']
self.assertEqual(len(entries), 3)
self.assertTrue(entries[0] is None)
self.assertTrue(entries[1] is None)
self.assertEqual(len(ydl.downloaded_info_dicts), 1)
downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(entries[2], downloaded)
self.assertEqual(downloaded['url'], TEST_URL)
self.assertEqual(downloaded['title'], 'Video Transparent 2')
self.assertEqual(downloaded['id'], '2')
self.assertEqual(downloaded['extractor'], 'Video')
self.assertEqual(downloaded['extractor_key'], 'Video')
def test_default_times(self):
"""Test addition of missing upload/release/_date from /release_/timestamp"""
info = {
'id': '1234',
'url': TEST_URL,
'title': 'Title',
'ext': 'mp4',
'timestamp': 1631352900,
'release_timestamp': 1632995931,
}
params = {'simulate': True, }
ydl = FakeYDL(params)
out_info = ydl.process_ie_result(info)
self.assertTrue(isinstance(out_info['upload_date'], compat_str))
self.assertEqual(out_info['upload_date'], '20210911')
self.assertTrue(isinstance(out_info['release_date'], compat_str))
self.assertEqual(out_info['release_date'], '20210930')
class TestYoutubeDLCookies(unittest.TestCase):
@staticmethod
def encode_cookie(cookie):
if not isinstance(cookie, dict):
cookie = vars(cookie)
for name, value in cookie.items():
yield name, compat_str(value)
@classmethod
def comparable_cookies(cls, cookies):
# Work around cookiejar cookies not being unicode strings
return sorted(map(tuple, map(sorted, map(cls.encode_cookie, cookies))))
def assertSameCookies(self, c1, c2, msg=None):
return self.assertEqual(
*map(self.comparable_cookies, (c1, c2)),
msg=msg)
def assertSameCookieStrings(self, c1, c2, msg=None):
return self.assertSameCookies(
*map(lambda c: compat_http_cookies_SimpleCookie(c).values(), (c1, c2)),
msg=msg)
def test_header_cookies(self):
ydl = FakeYDL()
ydl.report_warning = lambda *_, **__: None
def cookie(name, value, version=None, domain='', path='', secure=False, expires=None):
return compat_http_cookiejar_Cookie(
version or 0, name, value, None, False,
domain, bool(domain), bool(domain), path, bool(path),
secure, expires, False, None, None, rest={})
test_url, test_domain = (t % ('yt.dl',) for t in ('https://%s/test', '.%s'))
def test(encoded_cookies, cookies, headers=False, round_trip=None, error_re=None):
def _test():
ydl.cookiejar.clear()
ydl._load_cookies(encoded_cookies, autoscope=headers)
if headers:
ydl._apply_header_cookies(test_url)
data = {'url': test_url}
ydl._calc_headers(data)
self.assertSameCookies(
cookies, ydl.cookiejar,
'Extracted cookiejar.Cookie is not the same')
if not headers:
self.assertSameCookieStrings(
data.get('cookies'), round_trip or encoded_cookies,
msg='Cookie is not the same as round trip')
ydl.__dict__['_YoutubeDL__header_cookies'] = []
try:
_test()
except AssertionError:
raise
except Exception as e:
if not error_re:
raise
assertRegexpMatches(self, e.args[0], error_re.join(('.*',) * 2))
test('test=value; Domain=' + test_domain, [cookie('test', 'value', domain=test_domain)])
test('test=value', [cookie('test', 'value')], error_re='Unscoped cookies are not allowed')
test('cookie1=value1; Domain={0}; Path=/test; cookie2=value2; Domain={0}; Path=/'.format(test_domain), [
cookie('cookie1', 'value1', domain=test_domain, path='/test'),
cookie('cookie2', 'value2', domain=test_domain, path='/')])
cookie_kw = compat_kwargs(
{'domain': test_domain, 'path': '/test', 'secure': True, 'expires': '9999999999', })
test('test=value; Domain={domain}; Path={path}; Secure; Expires={expires}'.format(**cookie_kw), [
cookie('test', 'value', **cookie_kw)])
test('test="value; "; path=/test; domain=' + test_domain, [
cookie('test', 'value; ', domain=test_domain, path='/test')],
round_trip='test="value\\073 "; Domain={0}; Path=/test'.format(test_domain))
test('name=; Domain=' + test_domain, [cookie('name', '', domain=test_domain)],
round_trip='name=""; Domain=' + test_domain)
test('test=value', [cookie('test', 'value', domain=test_domain)], headers=True)
test('cookie1=value; Domain={0}; cookie2=value'.format(test_domain), [],
headers=True, error_re='Invalid syntax')
ydl.report_warning = ydl.report_error
test('test=value', [], headers=True, error_re='Passing cookies as a header is a potential security risk')
def test_infojson_cookies(self):
TEST_FILE = 'test_infojson_cookies.info.json'
TEST_URL = 'https://example.com/example.mp4'
COOKIES = 'a=b; Domain=.example.com; c=d; Domain=.example.com'
COOKIE_HEADER = {'Cookie': 'a=b; c=d'}
ydl = FakeYDL()
ydl.process_info = lambda x: ydl._write_info_json('test', x, TEST_FILE)
def make_info(info_header_cookies=False, fmts_header_cookies=False, cookies_field=False):
fmt = {'url': TEST_URL}
if fmts_header_cookies:
fmt['http_headers'] = COOKIE_HEADER
if cookies_field:
fmt['cookies'] = COOKIES
return _make_result([fmt], http_headers=COOKIE_HEADER if info_header_cookies else None)
def test(initial_info, note):
def failure_msg(why):
return ' when '.join((why, note))
result = {}
result['processed'] = ydl.process_ie_result(initial_info)
self.assertTrue(ydl.cookiejar.get_cookies_for_url(TEST_URL),
msg=failure_msg('No cookies set in cookiejar after initial process'))
ydl.cookiejar.clear()
with open(TEST_FILE) as infojson:
result['loaded'] = ydl.sanitize_info(json.load(infojson), True)
result['final'] = ydl.process_ie_result(result['loaded'].copy(), download=False)
self.assertTrue(ydl.cookiejar.get_cookies_for_url(TEST_URL),
msg=failure_msg('No cookies set in cookiejar after final process'))
ydl.cookiejar.clear()
for key in ('processed', 'loaded', 'final'):
info = result[key]
self.assertIsNone(
traverse_obj(info, ((None, ('formats', 0)), 'http_headers', 'Cookie'), casesense=False, get_all=False),
msg=failure_msg('Cookie header not removed in {0} result'.format(key)))
self.assertSameCookieStrings(
traverse_obj(info, ((None, ('formats', 0)), 'cookies'), get_all=False), COOKIES,
msg=failure_msg('No cookies field found in {0} result'.format(key)))
test({'url': TEST_URL, 'http_headers': COOKIE_HEADER, 'id': '1', 'title': 'x'}, 'no formats field')
test(make_info(info_header_cookies=True), 'info_dict header cokies')
test(make_info(fmts_header_cookies=True), 'format header cookies')
test(make_info(info_header_cookies=True, fmts_header_cookies=True), 'info_dict and format header cookies')
test(make_info(info_header_cookies=True, fmts_header_cookies=True, cookies_field=True), 'all cookies fields')
test(make_info(cookies_field=True), 'cookies format field')
test({'url': TEST_URL, 'cookies': COOKIES, 'id': '1', 'title': 'x'}, 'info_dict cookies field only')
try_rm(TEST_FILE)
def test_add_headers_cookie(self):
def check_for_cookie_header(result):
return traverse_obj(result, ((None, ('formats', 0)), 'http_headers', 'Cookie'), casesense=False, get_all=False)
ydl = FakeYDL({'http_headers': {'Cookie': 'a=b'}})
ydl._apply_header_cookies(_make_result([])['webpage_url']) # Scope to input webpage URL: .example.com
fmt = {'url': 'https://example.com/video.mp4'}
result = ydl.process_ie_result(_make_result([fmt]), download=False)
self.assertIsNone(check_for_cookie_header(result), msg='http_headers cookies in result info_dict')
self.assertEqual(result.get('cookies'), 'a=b; Domain=.example.com', msg='No cookies were set in cookies field')
self.assertIn('a=b', ydl.cookiejar.get_cookie_header(fmt['url']), msg='No cookies were set in cookiejar')
fmt = {'url': 'https://wrong.com/video.mp4'}
result = ydl.process_ie_result(_make_result([fmt]), download=False)
self.assertIsNone(check_for_cookie_header(result), msg='http_headers cookies for wrong domain')
self.assertFalse(result.get('cookies'), msg='Cookies set in cookies field for wrong domain')
self.assertFalse(ydl.cookiejar.get_cookie_header(fmt['url']), msg='Cookies set in cookiejar for wrong domain')
if __name__ == '__main__':

View File

@ -0,0 +1,65 @@
#!/usr/bin/env python
# coding: utf-8
from __future__ import unicode_literals
import os
import re
import sys
import tempfile
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from youtube_dl.utils import YoutubeDLCookieJar
class TestYoutubeDLCookieJar(unittest.TestCase):
def test_keep_session_cookies(self):
cookiejar = YoutubeDLCookieJar('./test/testdata/cookies/session_cookies.txt')
cookiejar.load(ignore_discard=True, ignore_expires=True)
tf = tempfile.NamedTemporaryFile(delete=False)
try:
cookiejar.save(filename=tf.name, ignore_discard=True, ignore_expires=True)
temp = tf.read().decode('utf-8')
self.assertTrue(re.search(
r'www\.foobar\.foobar\s+FALSE\s+/\s+TRUE\s+0\s+YoutubeDLExpiresEmpty\s+YoutubeDLExpiresEmptyValue', temp))
self.assertTrue(re.search(
r'www\.foobar\.foobar\s+FALSE\s+/\s+TRUE\s+0\s+YoutubeDLExpires0\s+YoutubeDLExpires0Value', temp))
finally:
tf.close()
os.remove(tf.name)
def test_strip_httponly_prefix(self):
cookiejar = YoutubeDLCookieJar('./test/testdata/cookies/httponly_cookies.txt')
cookiejar.load(ignore_discard=True, ignore_expires=True)
def assert_cookie_has_value(key):
self.assertEqual(cookiejar._cookies['www.foobar.foobar']['/'][key].value, key + '_VALUE')
assert_cookie_has_value('HTTPONLY_COOKIE')
assert_cookie_has_value('JS_ACCESSIBLE_COOKIE')
def test_malformed_cookies(self):
cookiejar = YoutubeDLCookieJar('./test/testdata/cookies/malformed_cookies.txt')
cookiejar.load(ignore_discard=True, ignore_expires=True)
# Cookies should be empty since all malformed cookie file entries
# will be ignored
self.assertFalse(cookiejar._cookies)
def test_get_cookie_header(self):
cookiejar = YoutubeDLCookieJar('./test/testdata/cookies/httponly_cookies.txt')
cookiejar.load(ignore_discard=True, ignore_expires=True)
header = cookiejar.get_cookie_header('https://www.foobar.foobar')
self.assertIn('HTTPONLY_COOKIE', header)
def test_get_cookies_for_url(self):
cookiejar = YoutubeDLCookieJar('./test/testdata/cookies/session_cookies.txt')
cookiejar.load(ignore_discard=True, ignore_expires=True)
cookies = cookiejar.get_cookies_for_url('https://www.foobar.foobar/')
self.assertEqual(len(cookies), 2)
cookies = cookiejar.get_cookies_for_url('https://foobar.foobar/')
self.assertFalse(cookies)
if __name__ == '__main__':
unittest.main()

View File

@ -8,7 +8,7 @@ import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from youtube_dl.aes import aes_decrypt, aes_encrypt, aes_cbc_decrypt, aes_decrypt_text
from youtube_dl.aes import aes_decrypt, aes_encrypt, aes_cbc_decrypt, aes_cbc_encrypt, aes_decrypt_text, aes_ecb_encrypt
from youtube_dl.utils import bytes_to_intlist, intlist_to_bytes
import base64
@ -34,22 +34,37 @@ class TestAES(unittest.TestCase):
decrypted = intlist_to_bytes(aes_cbc_decrypt(data, self.key, self.iv))
self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
def test_cbc_encrypt(self):
data = bytes_to_intlist(self.secret_msg)
encrypted = intlist_to_bytes(aes_cbc_encrypt(data, self.key, self.iv))
self.assertEqual(
encrypted,
b"\x97\x92+\xe5\x0b\xc3\x18\x91ky9m&\xb3\xb5@\xe6'\xc2\x96.\xc8u\x88\xab9-[\x9e|\xf1\xcd")
def test_decrypt_text(self):
password = intlist_to_bytes(self.key).decode('utf-8')
encrypted = base64.b64encode(
intlist_to_bytes(self.iv[:8]) +
b'\x17\x15\x93\xab\x8d\x80V\xcdV\xe0\t\xcdo\xc2\xa5\xd8ksM\r\xe27N\xae'
intlist_to_bytes(self.iv[:8])
+ b'\x17\x15\x93\xab\x8d\x80V\xcdV\xe0\t\xcdo\xc2\xa5\xd8ksM\r\xe27N\xae'
).decode('utf-8')
decrypted = (aes_decrypt_text(encrypted, password, 16))
self.assertEqual(decrypted, self.secret_msg)
password = intlist_to_bytes(self.key).decode('utf-8')
encrypted = base64.b64encode(
intlist_to_bytes(self.iv[:8]) +
b'\x0b\xe6\xa4\xd9z\x0e\xb8\xb9\xd0\xd4i_\x85\x1d\x99\x98_\xe5\x80\xe7.\xbf\xa5\x83'
intlist_to_bytes(self.iv[:8])
+ b'\x0b\xe6\xa4\xd9z\x0e\xb8\xb9\xd0\xd4i_\x85\x1d\x99\x98_\xe5\x80\xe7.\xbf\xa5\x83'
).decode('utf-8')
decrypted = (aes_decrypt_text(encrypted, password, 32))
self.assertEqual(decrypted, self.secret_msg)
def test_ecb_encrypt(self):
data = bytes_to_intlist(self.secret_msg)
encrypted = intlist_to_bytes(aes_ecb_encrypt(data, self.key))
self.assertEqual(
encrypted,
b'\xaa\x86]\x81\x97>\x02\x92\x9d\x1bR[[L/u\xd3&\xd1(h\xde{\x81\x94\xba\x02\xae\xbd\xa6\xd0:')
if __name__ == '__main__':
unittest.main()

View File

@ -11,6 +11,7 @@ from test.helper import try_rm
from youtube_dl import YoutubeDL
from youtube_dl.utils import DownloadError
def _download_restricted(url, filename, age):
@ -26,7 +27,10 @@ def _download_restricted(url, filename, age):
ydl.add_default_info_extractors()
json_filename = os.path.splitext(filename)[0] + '.info.json'
try_rm(json_filename)
ydl.download([url])
try:
ydl.download([url])
except DownloadError:
try_rm(json_filename)
res = os.path.exists(json_filename)
try_rm(json_filename)
return res
@ -38,12 +42,12 @@ class TestAgeRestriction(unittest.TestCase):
self.assertFalse(_download_restricted(url, filename, age))
def test_youtube(self):
self._assert_restricted('07FYdnEawAQ', '07FYdnEawAQ.mp4', 10)
self._assert_restricted('HtVdAasjOgU', 'HtVdAasjOgU.mp4', 10)
def test_youporn(self):
self._assert_restricted(
'http://www.youporn.com/watch/505835/sex-ed-is-it-safe-to-masturbate-daily/',
'505835.mp4', 2, old_age=25)
'https://www.youporn.com/watch/16715086/sex-ed-in-detention-18-asmr/',
'16715086.mp4', 2, old_age=25)
if __name__ == '__main__':

View File

@ -6,6 +6,7 @@ from __future__ import unicode_literals
import os
import sys
import unittest
import collections
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
@ -30,16 +31,17 @@ class TestAllURLsMatching(unittest.TestCase):
def test_youtube_playlist_matching(self):
assertPlaylist = lambda url: self.assertMatch(url, ['youtube:playlist'])
assertTab = lambda url: self.assertMatch(url, ['youtube:tab'])
assertPlaylist('ECUl4u3cNGP61MdtwGTqZA0MreSaDybji8')
assertPlaylist('UUBABnxM4Ar9ten8Mdjj1j0Q') # 585
assertPlaylist('PL63F0C78739B09958')
assertPlaylist('https://www.youtube.com/playlist?list=UUBABnxM4Ar9ten8Mdjj1j0Q')
assertPlaylist('https://www.youtube.com/course?list=ECUl4u3cNGP61MdtwGTqZA0MreSaDybji8')
assertPlaylist('https://www.youtube.com/playlist?list=PLwP_SiAcdui0KVebT0mU9Apz359a4ubsC')
assertPlaylist('https://www.youtube.com/watch?v=AV6J6_AeFEQ&playnext=1&list=PL4023E734DA416012') # 668
assertTab('https://www.youtube.com/playlist?list=UUBABnxM4Ar9ten8Mdjj1j0Q')
assertTab('https://www.youtube.com/course?list=ECUl4u3cNGP61MdtwGTqZA0MreSaDybji8')
assertTab('https://www.youtube.com/playlist?list=PLwP_SiAcdui0KVebT0mU9Apz359a4ubsC')
assertTab('https://www.youtube.com/watch?v=AV6J6_AeFEQ&playnext=1&list=PL4023E734DA416012') # 668
self.assertFalse('youtube:playlist' in self.matching_ies('PLtS2H6bU1M'))
# Top tracks
assertPlaylist('https://www.youtube.com/playlist?list=MCUS.20142101')
assertTab('https://www.youtube.com/playlist?list=MCUS.20142101')
def test_youtube_matching(self):
self.assertTrue(YoutubeIE.suitable('PLtS2H6bU1M'))
@ -50,36 +52,24 @@ class TestAllURLsMatching(unittest.TestCase):
self.assertMatch('http://www.cleanvideosearch.com/media/action/yt/watch?videoId=8v_4O44sfjM', ['youtube'])
def test_youtube_channel_matching(self):
assertChannel = lambda url: self.assertMatch(url, ['youtube:channel'])
assertChannel = lambda url: self.assertMatch(url, ['youtube:tab'])
assertChannel('https://www.youtube.com/channel/HCtnHdj3df7iM')
assertChannel('https://www.youtube.com/channel/HCtnHdj3df7iM?feature=gb_ch_rec')
assertChannel('https://www.youtube.com/channel/HCtnHdj3df7iM/videos')
def test_youtube_user_matching(self):
self.assertMatch('http://www.youtube.com/NASAgovVideo/videos', ['youtube:user'])
self.assertMatch('http://www.youtube.com/NASAgovVideo/videos', ['youtube:tab'])
def test_youtube_feeds(self):
self.assertMatch('https://www.youtube.com/feed/watch_later', ['youtube:watchlater'])
self.assertMatch('https://www.youtube.com/feed/subscriptions', ['youtube:subscriptions'])
self.assertMatch('https://www.youtube.com/feed/recommended', ['youtube:recommended'])
self.assertMatch('https://www.youtube.com/my_favorites', ['youtube:favorites'])
def test_youtube_show_matching(self):
self.assertMatch('http://www.youtube.com/show/airdisasters', ['youtube:show'])
self.assertMatch('https://www.youtube.com/feed/library', ['youtube:tab'])
self.assertMatch('https://www.youtube.com/feed/history', ['youtube:tab'])
self.assertMatch('https://www.youtube.com/feed/watch_later', ['youtube:tab'])
self.assertMatch('https://www.youtube.com/feed/subscriptions', ['youtube:tab'])
def test_youtube_search_matching(self):
self.assertMatch('http://www.youtube.com/results?search_query=making+mustard', ['youtube:search_url'])
self.assertMatch('https://www.youtube.com/results?baz=bar&search_query=youtube-dl+test+video&filters=video&lclk=video', ['youtube:search_url'])
def test_youtube_extract(self):
assertExtractId = lambda url, id: self.assertEqual(YoutubeIE.extract_id(url), id)
assertExtractId('http://www.youtube.com/watch?&v=BaW_jenozKc', 'BaW_jenozKc')
assertExtractId('https://www.youtube.com/watch?&v=BaW_jenozKc', 'BaW_jenozKc')
assertExtractId('https://www.youtube.com/watch?feature=player_embedded&v=BaW_jenozKc', 'BaW_jenozKc')
assertExtractId('https://www.youtube.com/watch_popup?v=BaW_jenozKc', 'BaW_jenozKc')
assertExtractId('http://www.youtube.com/watch?v=BaW_jenozKcsharePLED17F32AD9753930', 'BaW_jenozKc')
assertExtractId('BaW_jenozKc', 'BaW_jenozKc')
def test_facebook_matching(self):
self.assertTrue(FacebookIE.suitable('https://www.facebook.com/Shiniknoh#!/photo.php?v=10153317450565268'))
self.assertTrue(FacebookIE.suitable('https://www.facebook.com/cindyweather?fref=ts#!/photo.php?v=10152183998945793'))
@ -100,8 +90,6 @@ class TestAllURLsMatching(unittest.TestCase):
self.assertMatch(':ytsubs', ['youtube:subscriptions'])
self.assertMatch(':ytsubscriptions', ['youtube:subscriptions'])
self.assertMatch(':ythistory', ['youtube:history'])
self.assertMatch(':thedailyshow', ['ComedyCentralShows'])
self.assertMatch(':tds', ['ComedyCentralShows'])
def test_vimeo_matching(self):
self.assertMatch('https://vimeo.com/channels/tributes', ['vimeo:channel'])
@ -111,7 +99,7 @@ class TestAllURLsMatching(unittest.TestCase):
self.assertMatch('https://vimeo.com/user7108434/videos', ['vimeo:user'])
self.assertMatch('https://vimeo.com/user21297594/review/75524534/3c257a1b5d', ['vimeo:review'])
# https://github.com/rg3/youtube-dl/issues/1930
# https://github.com/ytdl-org/youtube-dl/issues/1930
def test_soundcloud_not_matching_sets(self):
self.assertMatch('http://soundcloud.com/floex/sets/gone-ep', ['soundcloud:set'])
@ -120,15 +108,18 @@ class TestAllURLsMatching(unittest.TestCase):
self.assertMatch('http://tatianamaslanydaily.tumblr.com/post/54196191430', ['Tumblr'])
def test_pbs(self):
# https://github.com/rg3/youtube-dl/issues/2350
# https://github.com/ytdl-org/youtube-dl/issues/2350
self.assertMatch('http://video.pbs.org/viralplayer/2365173446/', ['pbs'])
self.assertMatch('http://video.pbs.org/widget/partnerplayer/980042464/', ['pbs'])
def test_yahoo_https(self):
# https://github.com/rg3/youtube-dl/issues/2701
self.assertMatch(
'https://screen.yahoo.com/smartwatches-latest-wearable-gadgets-163745379-cbs.html',
['Yahoo'])
def test_no_duplicated_ie_names(self):
name_accu = collections.defaultdict(list)
for ie in self.ies:
name_accu[ie.IE_NAME.lower()].append(type(ie).__name__)
for (ie_name, ie_list) in name_accu.items():
self.assertEqual(
len(ie_list), 1,
'Multiple extractors with the same IE_NAME "%s" (%s)' % (ie_name, ', '.join(ie_list)))
if __name__ == '__main__':

View File

@ -3,17 +3,18 @@
from __future__ import unicode_literals
import shutil
# Allow direct execution
import os
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import shutil
from test.helper import FakeYDL
from youtube_dl.cache import Cache
from youtube_dl.utils import version_tuple
from youtube_dl.version import __version__
def _is_empty(d):
@ -54,6 +55,29 @@ class TestCache(unittest.TestCase):
self.assertFalse(os.path.exists(self.test_dir))
self.assertEqual(c.load('test_cache', 'k.'), None)
def test_cache_validation(self):
ydl = FakeYDL({
'cachedir': self.test_dir,
})
c = Cache(ydl)
obj = {'x': 1, 'y': ['ä', '\\a', True]}
c.store('test_cache', 'k.', obj)
self.assertEqual(c.load('test_cache', 'k.', min_ver='1970.01.01'), obj)
new_version = '.'.join(('%0.2d' % ((v + 1) if i == 0 else v, )) for i, v in enumerate(version_tuple(__version__)))
self.assertIs(c.load('test_cache', 'k.', min_ver=new_version), None)
def test_cache_clear(self):
ydl = FakeYDL({
'cachedir': self.test_dir,
})
c = Cache(ydl)
c.store('test_cache', 'k.', 'kay')
c.store('test_cache', 'l.', 'ell')
self.assertEqual(c.load('test_cache', 'k.'), 'kay')
c.clear('test_cache', 'k.')
self.assertEqual(c.load('test_cache', 'k.'), None)
self.assertEqual(c.load('test_cache', 'l.'), 'ell')
if __name__ == '__main__':
unittest.main()

View File

@ -10,41 +10,50 @@ import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from youtube_dl.utils import get_filesystem_encoding
from youtube_dl.compat import (
compat_casefold,
compat_getenv,
compat_setenv,
compat_etree_Element,
compat_etree_fromstring,
compat_expanduser,
compat_shlex_split,
compat_str,
compat_struct_unpack,
compat_urllib_parse_unquote,
compat_urllib_parse_unquote_plus,
compat_urllib_parse_urlencode,
compat_urllib_request,
)
class TestCompat(unittest.TestCase):
def test_compat_getenv(self):
test_str = 'тест'
os.environ['YOUTUBE-DL-TEST'] = (
test_str if sys.version_info >= (3, 0)
else test_str.encode(get_filesystem_encoding()))
self.assertEqual(compat_getenv('YOUTUBE-DL-TEST'), test_str)
compat_setenv('YOUTUBE_DL_COMPAT_GETENV', test_str)
self.assertEqual(compat_getenv('YOUTUBE_DL_COMPAT_GETENV'), test_str)
def test_compat_setenv(self):
test_var = 'YOUTUBE_DL_COMPAT_SETENV'
test_str = 'тест'
compat_setenv(test_var, test_str)
compat_getenv(test_var)
self.assertEqual(compat_getenv(test_var), test_str)
def test_compat_expanduser(self):
old_home = os.environ.get('HOME')
test_str = 'C:\Documents and Settings\тест\Application Data'
os.environ['HOME'] = (
test_str if sys.version_info >= (3, 0)
else test_str.encode(get_filesystem_encoding()))
test_str = r'C:\Documents and Settings\тест\Application Data'
compat_setenv('HOME', test_str)
self.assertEqual(compat_expanduser('~'), test_str)
os.environ['HOME'] = old_home
compat_setenv('HOME', old_home or '')
def test_all_present(self):
import youtube_dl.compat
all_names = youtube_dl.compat.__all__
present_names = set(filter(
all_names = sorted(
youtube_dl.compat.__all__ + youtube_dl.compat.legacy)
present_names = set(map(compat_str, filter(
lambda c: '_' in c and not c.startswith('_'),
dir(youtube_dl.compat))) - set(['unicode_literals'])
dir(youtube_dl.compat)))) - set(['unicode_literals'])
self.assertEqual(all_names, sorted(present_names))
def test_compat_urllib_parse_unquote(self):
@ -70,8 +79,26 @@ class TestCompat(unittest.TestCase):
self.assertEqual(compat_urllib_parse_unquote_plus('abc%20def'), 'abc def')
self.assertEqual(compat_urllib_parse_unquote_plus('%7e/abc+def'), '~/abc def')
def test_compat_urllib_parse_urlencode(self):
self.assertEqual(compat_urllib_parse_urlencode({'abc': 'def'}), 'abc=def')
self.assertEqual(compat_urllib_parse_urlencode({'abc': b'def'}), 'abc=def')
self.assertEqual(compat_urllib_parse_urlencode({b'abc': 'def'}), 'abc=def')
self.assertEqual(compat_urllib_parse_urlencode({b'abc': b'def'}), 'abc=def')
self.assertEqual(compat_urllib_parse_urlencode([('abc', 'def')]), 'abc=def')
self.assertEqual(compat_urllib_parse_urlencode([('abc', b'def')]), 'abc=def')
self.assertEqual(compat_urllib_parse_urlencode([(b'abc', 'def')]), 'abc=def')
self.assertEqual(compat_urllib_parse_urlencode([(b'abc', b'def')]), 'abc=def')
def test_compat_shlex_split(self):
self.assertEqual(compat_shlex_split('-option "one two"'), ['-option', 'one two'])
self.assertEqual(compat_shlex_split('-option "one\ntwo" \n -flag'), ['-option', 'one\ntwo', '-flag'])
self.assertEqual(compat_shlex_split('-val 中文'), ['-val', '中文'])
def test_compat_etree_Element(self):
try:
compat_etree_Element.items
except AttributeError:
self.fail('compat_etree_Element is not a type')
def test_compat_etree_fromstring(self):
xml = '''
@ -88,5 +115,40 @@ class TestCompat(unittest.TestCase):
self.assertTrue(isinstance(doc.find('chinese').text, compat_str))
self.assertTrue(isinstance(doc.find('foo/bar').text, compat_str))
def test_compat_etree_fromstring_doctype(self):
xml = '''<?xml version="1.0"?>
<!DOCTYPE smil PUBLIC "-//W3C//DTD SMIL 2.0//EN" "http://www.w3.org/2001/SMIL20/SMIL20.dtd">
<smil xmlns="http://www.w3.org/2001/SMIL20/Language"></smil>'''
compat_etree_fromstring(xml)
def test_compat_struct_unpack(self):
self.assertEqual(compat_struct_unpack('!B', b'\x00'), (0,))
def test_compat_casefold(self):
if hasattr(compat_str, 'casefold'):
# don't bother to test str.casefold() (again)
return
# thanks https://bugs.python.org/file24232/casefolding.patch
self.assertEqual(compat_casefold('hello'), 'hello')
self.assertEqual(compat_casefold('hELlo'), 'hello')
self.assertEqual(compat_casefold('ß'), 'ss')
self.assertEqual(compat_casefold(''), 'fi')
self.assertEqual(compat_casefold('\u03a3'), '\u03c3')
self.assertEqual(compat_casefold('A\u0345\u03a3'), 'a\u03b9\u03c3')
def test_compat_urllib_request_Request(self):
self.assertEqual(
compat_urllib_request.Request('http://127.0.0.1', method='PUT').get_method(),
'PUT')
class PUTrequest(compat_urllib_request.Request):
def get_method(self):
return 'PUT'
self.assertEqual(
PUTrequest('http://127.0.0.1').get_method(),
'PUT')
if __name__ == '__main__':
unittest.main()

View File

@ -9,7 +9,6 @@ import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from test.helper import (
assertGreaterEqual,
expect_warnings,
get_params,
gettestcases,
@ -20,26 +19,35 @@ from test.helper import (
import hashlib
import io
import json
import socket
import youtube_dl.YoutubeDL
from youtube_dl.compat import (
compat_http_client,
compat_urllib_error,
compat_HTTPError,
compat_open as open,
compat_urllib_error,
)
from youtube_dl.utils import (
DownloadError,
ExtractorError,
error_to_compat_str,
format_bytes,
IDENTITY,
preferredencoding,
UnavailableVideoError,
)
from youtube_dl.extractor import get_info_extractor
RETRIES = 3
# Some unittest APIs require actual str
if not isinstance('TEST', str):
_encode_str = lambda s: s.encode(preferredencoding())
else:
_encode_str = IDENTITY
class YoutubeDL(youtube_dl.YoutubeDL):
def __init__(self, *args, **kwargs):
@ -60,49 +68,71 @@ def _file_md5(fn):
with open(fn, 'rb') as f:
return hashlib.md5(f.read()).hexdigest()
defs = gettestcases()
class TestDownload(unittest.TestCase):
# Parallel testing in nosetests. See
# http://nose.readthedocs.org/en/latest/doc_tests/test_multiprocess/multiprocess.html
_multiprocess_shared_ = True
maxDiff = None
def __str__(self):
"""Identify each test with the `add_ie` attribute, if available."""
def strclass(cls):
"""From 2.7's unittest; 2.6 had _strclass so we can't import it."""
return '%s.%s' % (cls.__module__, cls.__name__)
add_ie = getattr(self, self._testMethodName).add_ie
return '%s (%s)%s:' % (self._testMethodName,
strclass(self.__class__),
' [%s]' % add_ie if add_ie else '')
def setUp(self):
self.defs = defs
# Dynamically generate tests
def generator(test_case):
def generator(test_case, tname):
def test_template(self):
ie = youtube_dl.extractor.get_info_extractor(test_case['name'])
other_ies = [get_info_extractor(ie_key) for ie_key in test_case.get('add_ie', [])]
ie = youtube_dl.extractor.get_info_extractor(test_case['name'])()
other_ies = [get_info_extractor(ie_key)() for ie_key in test_case.get('add_ie', [])]
is_playlist = any(k.startswith('playlist') for k in test_case)
test_cases = test_case.get(
'playlist', [] if is_playlist else [test_case])
def print_skipping(reason):
print('Skipping %s: %s' % (test_case['name'], reason))
self.skipTest(_encode_str(reason))
if not ie.working():
print_skipping('IE marked as not _WORKING')
return
for tc in test_cases:
info_dict = tc.get('info_dict', {})
if not (info_dict.get('id') and info_dict.get('ext')):
raise Exception('Test definition incorrect. The output file cannot be known. Are both \'id\' and \'ext\' keys present?')
raise Exception('Test definition (%s) requires both \'id\' and \'ext\' keys present to define the output file' % (tname, ))
if 'skip' in test_case:
print_skipping(test_case['skip'])
return
for other_ie in other_ies:
if not other_ie.working():
print_skipping('test depends on %sIE, marked as not WORKING' % other_ie.ie_key())
return
params = get_params(test_case.get('params', {}))
params['outtmpl'] = tname + '_' + params['outtmpl']
if is_playlist and 'playlist' not in test_case:
params.setdefault('extract_flat', 'in_playlist')
params.setdefault('playlistend',
test_case['playlist_maxcount'] + 1
if test_case.get('playlist_maxcount')
else test_case.get('playlist_mincount'))
params.setdefault('skip_download', True)
ydl = YoutubeDL(params, auto_init=False)
@ -128,12 +158,13 @@ def generator(test_case):
try_rm(tc_filename)
try_rm(tc_filename + '.part')
try_rm(os.path.splitext(tc_filename)[0] + '.info.json')
try_rm_tcs_files()
try:
try_num = 1
while True:
try:
# We're not using .download here sine that is just a shim
# We're not using .download here since that is just a shim
# for outside error handling, and returns the exit code
# instead of the result dict.
res_dict = ydl.extract_info(
@ -142,10 +173,12 @@ def generator(test_case):
except (DownloadError, ExtractorError) as err:
# Check if the exception is not a network related one
if not err.exc_info[0] in (compat_urllib_error.URLError, socket.timeout, UnavailableVideoError, compat_http_client.BadStatusLine) or (err.exc_info[0] == compat_HTTPError and err.exc_info[1].code == 503):
raise
msg = getattr(err, 'msg', error_to_compat_str(err))
err.msg = '%s (%s)' % (msg, tname, )
raise err
if try_num == RETRIES:
report_warning('Failed due to network errors, skipping...')
report_warning('%s failed due to network errors, skipping...' % tname)
return
print('Retrying: {0} failed tries\n\n##########\n\n'.format(try_num))
@ -160,13 +193,19 @@ def generator(test_case):
expect_info_dict(self, res_dict, test_case.get('info_dict', {}))
if 'playlist_mincount' in test_case:
assertGreaterEqual(
self,
self.assertGreaterEqual(
len(res_dict['entries']),
test_case['playlist_mincount'],
'Expected at least %d in playlist %s, but got only %d' % (
test_case['playlist_mincount'], test_case['url'],
len(res_dict['entries'])))
if 'playlist_maxcount' in test_case:
self.assertLessEqual(
len(res_dict['entries']),
test_case['playlist_maxcount'],
'Expected at most %d in playlist %s, but got %d' % (
test_case['playlist_maxcount'], test_case['url'],
len(res_dict['entries'])))
if 'playlist_count' in test_case:
self.assertEqual(
len(res_dict['entries']),
@ -181,8 +220,25 @@ def generator(test_case):
self.assertEqual(
test_case['playlist_duration_sum'], got_duration)
for tc in test_cases:
# Generalize both playlists and single videos to unified format for
# simplicity
if 'entries' not in res_dict:
res_dict['entries'] = [res_dict]
for tc_num, tc in enumerate(test_cases):
tc_res_dict = res_dict['entries'][tc_num]
# First, check test cases' data against extracted data alone
expect_info_dict(self, tc_res_dict, tc.get('info_dict', {}))
# Now, check downloaded file consistency
# support test-case with volatile ID, signalled by regexp value
if tc.get('info_dict', {}).get('id', '').startswith('re:'):
test_id = tc['info_dict']['id']
tc['info_dict']['id'] = tc_res_dict['id']
else:
test_id = None
tc_filename = get_tc_filename(tc)
if test_id:
tc['info_dict']['id'] = test_id
if not test_case.get('params', {}).get('skip_download', False):
self.assertTrue(os.path.exists(tc_filename), msg='Missing file ' + tc_filename)
self.assertTrue(tc_filename in finished_hook_called)
@ -191,21 +247,22 @@ def generator(test_case):
if params.get('test'):
expected_minsize = max(expected_minsize, 10000)
got_fsize = os.path.getsize(tc_filename)
assertGreaterEqual(
self, got_fsize, expected_minsize,
self.assertGreaterEqual(
got_fsize, expected_minsize,
'Expected %s to be at least %s, but it\'s only %s ' %
(tc_filename, format_bytes(expected_minsize),
format_bytes(got_fsize)))
if 'md5' in tc:
md5_for_file = _file_md5(tc_filename)
self.assertEqual(md5_for_file, tc['md5'])
self.assertEqual(tc['md5'], md5_for_file)
# Finally, check test cases' data again but this time against
# extracted data from info JSON file written during processing
info_json_fn = os.path.splitext(tc_filename)[0] + '.info.json'
self.assertTrue(
os.path.exists(info_json_fn),
'Missing info file %s' % info_json_fn)
with io.open(info_json_fn, encoding='utf-8') as infof:
with open(info_json_fn, encoding='utf-8') as infof:
info_dict = json.load(infof)
expect_info_dict(self, info_dict, tc.get('info_dict', {}))
finally:
try_rm_tcs_files()
@ -217,15 +274,18 @@ def generator(test_case):
return test_template
# And add them to TestDownload
for n, test_case in enumerate(defs):
test_method = generator(test_case)
tname = 'test_' + str(test_case['name'])
i = 1
while hasattr(TestDownload, tname):
tname = 'test_%s_%d' % (test_case['name'], i)
i += 1
test_method = generator(test_case, tname)
test_method.__name__ = str(tname)
ie_list = test_case.get('add_ie')
test_method.add_ie = ie_list and ','.join(ie_list)
setattr(TestDownload, test_method.__name__, test_method)
del test_method

View File

@ -0,0 +1,272 @@
#!/usr/bin/env python
# coding: utf-8
from __future__ import unicode_literals
# Allow direct execution
import os
import re
import sys
import subprocess
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from test.helper import (
FakeLogger,
FakeYDL,
http_server_port,
try_rm,
)
from youtube_dl import YoutubeDL
from youtube_dl.compat import (
compat_contextlib_suppress,
compat_http_cookiejar_Cookie,
compat_http_server,
compat_kwargs,
)
from youtube_dl.utils import (
encodeFilename,
join_nonempty,
)
from youtube_dl.downloader.external import (
Aria2cFD,
Aria2pFD,
AxelFD,
CurlFD,
FFmpegFD,
HttpieFD,
WgetFD,
)
from youtube_dl.postprocessor import (
FFmpegPostProcessor,
)
import threading
TEST_SIZE = 10 * 1024
TEST_COOKIE = {
'version': 0,
'name': 'test',
'value': 'ytdlp',
'port': None,
'port_specified': False,
'domain': '.example.com',
'domain_specified': True,
'domain_initial_dot': False,
'path': '/',
'path_specified': True,
'secure': False,
'expires': None,
'discard': False,
'comment': None,
'comment_url': None,
'rest': {},
}
TEST_COOKIE_VALUE = join_nonempty('name', 'value', delim='=', from_dict=TEST_COOKIE)
TEST_INFO = {'url': 'http://www.example.com/'}
def cookiejar_Cookie(**cookie_args):
return compat_http_cookiejar_Cookie(**compat_kwargs(cookie_args))
def ifExternalFDAvailable(externalFD):
return unittest.skipUnless(externalFD.available(),
externalFD.get_basename() + ' not found')
class HTTPTestRequestHandler(compat_http_server.BaseHTTPRequestHandler):
def log_message(self, format, *args):
pass
def send_content_range(self, total=None):
range_header = self.headers.get('Range')
start = end = None
if range_header:
mobj = re.match(r'bytes=(\d+)-(\d+)', range_header)
if mobj:
start, end = (int(mobj.group(i)) for i in (1, 2))
valid_range = start is not None and end is not None
if valid_range:
content_range = 'bytes %d-%d' % (start, end)
if total:
content_range += '/%d' % total
self.send_header('Content-Range', content_range)
return (end - start + 1) if valid_range else total
def serve(self, range=True, content_length=True):
self.send_response(200)
self.send_header('Content-Type', 'video/mp4')
size = TEST_SIZE
if range:
size = self.send_content_range(TEST_SIZE)
if content_length:
self.send_header('Content-Length', size)
self.end_headers()
self.wfile.write(b'#' * size)
def do_GET(self):
if self.path == '/regular':
self.serve()
elif self.path == '/no-content-length':
self.serve(content_length=False)
elif self.path == '/no-range':
self.serve(range=False)
elif self.path == '/no-range-no-content-length':
self.serve(range=False, content_length=False)
else:
assert False, 'unrecognised server path'
@ifExternalFDAvailable(Aria2pFD)
class TestAria2pFD(unittest.TestCase):
def setUp(self):
self.httpd = compat_http_server.HTTPServer(
('127.0.0.1', 0), HTTPTestRequestHandler)
self.port = http_server_port(self.httpd)
self.server_thread = threading.Thread(target=self.httpd.serve_forever)
self.server_thread.daemon = True
self.server_thread.start()
def download(self, params, ep):
with subprocess.Popen(
['aria2c', '--enable-rpc'],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL
) as process:
if not process.poll():
filename = 'testfile.mp4'
params['logger'] = FakeLogger()
params['outtmpl'] = filename
ydl = YoutubeDL(params)
try_rm(encodeFilename(filename))
self.assertEqual(ydl.download(['http://127.0.0.1:%d/%s' % (self.port, ep)]), 0)
self.assertEqual(os.path.getsize(encodeFilename(filename)), TEST_SIZE)
try_rm(encodeFilename(filename))
process.kill()
def download_all(self, params):
for ep in ('regular', 'no-content-length', 'no-range', 'no-range-no-content-length'):
self.download(params, ep)
def test_regular(self):
self.download_all({'external_downloader': 'aria2p'})
def test_chunked(self):
self.download_all({
'external_downloader': 'aria2p',
'http_chunk_size': 1000,
})
@ifExternalFDAvailable(HttpieFD)
class TestHttpieFD(unittest.TestCase):
def test_make_cmd(self):
with FakeYDL() as ydl:
downloader = HttpieFD(ydl, {})
self.assertEqual(
downloader._make_cmd('test', TEST_INFO),
['http', '--download', '--output', 'test', 'http://www.example.com/'])
# Test cookie header is added
ydl.cookiejar.set_cookie(cookiejar_Cookie(**TEST_COOKIE))
self.assertEqual(
downloader._make_cmd('test', TEST_INFO),
['http', '--download', '--output', 'test',
'http://www.example.com/', 'Cookie:' + TEST_COOKIE_VALUE])
@ifExternalFDAvailable(AxelFD)
class TestAxelFD(unittest.TestCase):
def test_make_cmd(self):
with FakeYDL() as ydl:
downloader = AxelFD(ydl, {})
self.assertEqual(
downloader._make_cmd('test', TEST_INFO),
['axel', '-o', 'test', '--', 'http://www.example.com/'])
# Test cookie header is added
ydl.cookiejar.set_cookie(cookiejar_Cookie(**TEST_COOKIE))
self.assertEqual(
downloader._make_cmd('test', TEST_INFO),
['axel', '-o', 'test', '-H', 'Cookie: ' + TEST_COOKIE_VALUE,
'--max-redirect=0', '--', 'http://www.example.com/'])
@ifExternalFDAvailable(WgetFD)
class TestWgetFD(unittest.TestCase):
def test_make_cmd(self):
with FakeYDL() as ydl:
downloader = WgetFD(ydl, {})
self.assertNotIn('--load-cookies', downloader._make_cmd('test', TEST_INFO))
# Test cookiejar tempfile arg is added
ydl.cookiejar.set_cookie(cookiejar_Cookie(**TEST_COOKIE))
self.assertIn('--load-cookies', downloader._make_cmd('test', TEST_INFO))
@ifExternalFDAvailable(CurlFD)
class TestCurlFD(unittest.TestCase):
def test_make_cmd(self):
with FakeYDL() as ydl:
downloader = CurlFD(ydl, {})
self.assertNotIn('--cookie', downloader._make_cmd('test', TEST_INFO))
# Test cookie header is added
ydl.cookiejar.set_cookie(cookiejar_Cookie(**TEST_COOKIE))
self.assertIn('--cookie', downloader._make_cmd('test', TEST_INFO))
self.assertIn(TEST_COOKIE_VALUE, downloader._make_cmd('test', TEST_INFO))
@ifExternalFDAvailable(Aria2cFD)
class TestAria2cFD(unittest.TestCase):
def test_make_cmd(self):
with FakeYDL() as ydl:
downloader = Aria2cFD(ydl, {})
downloader._make_cmd('test', TEST_INFO)
self.assertFalse(hasattr(downloader, '_cookies_tempfile'))
# Test cookiejar tempfile arg is added
ydl.cookiejar.set_cookie(cookiejar_Cookie(**TEST_COOKIE))
cmd = downloader._make_cmd('test', TEST_INFO)
self.assertIn('--load-cookies=%s' % downloader._cookies_tempfile, cmd)
# Handle delegated availability
def ifFFmpegFDAvailable(externalFD):
# raise SkipTest, or set False!
avail = ifExternalFDAvailable(externalFD) and False
with compat_contextlib_suppress(Exception):
avail = FFmpegPostProcessor(downloader=None).available
return unittest.skipUnless(
avail, externalFD.get_basename() + ' not found')
@ifFFmpegFDAvailable(FFmpegFD)
class TestFFmpegFD(unittest.TestCase):
_args = []
def _test_cmd(self, args):
self._args = args
def test_make_cmd(self):
with FakeYDL() as ydl:
downloader = FFmpegFD(ydl, {})
downloader._debug_cmd = self._test_cmd
info_dict = TEST_INFO.copy()
info_dict['ext'] = 'mp4'
downloader._call_downloader('test', info_dict)
self.assertEqual(self._args, [
'ffmpeg', '-y', '-i', 'http://www.example.com/',
'-c', 'copy', '-f', 'mp4', 'file:test'])
# Test cookies arg is added
ydl.cookiejar.set_cookie(cookiejar_Cookie(**TEST_COOKIE))
downloader._call_downloader('test', info_dict)
self.assertEqual(self._args, [
'ffmpeg', '-y', '-cookies', TEST_COOKIE_VALUE + '; path=/; domain=.example.com;\r\n',
'-i', 'http://www.example.com/', '-c', 'copy', '-f', 'mp4', 'file:test'])
if __name__ == '__main__':
unittest.main()

View File

@ -0,0 +1,108 @@
#!/usr/bin/env python
# coding: utf-8
from __future__ import unicode_literals
# Allow direct execution
import os
import re
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from test.helper import (
FakeLogger,
http_server_port,
try_rm,
)
from youtube_dl import YoutubeDL
from youtube_dl.compat import compat_http_server
from youtube_dl.downloader.http import HttpFD
from youtube_dl.utils import encodeFilename
import threading
TEST_DIR = os.path.dirname(os.path.abspath(__file__))
TEST_SIZE = 10 * 1024
class HTTPTestRequestHandler(compat_http_server.BaseHTTPRequestHandler):
def log_message(self, format, *args):
pass
def send_content_range(self, total=None):
range_header = self.headers.get('Range')
start = end = None
if range_header:
mobj = re.search(r'^bytes=(\d+)-(\d+)', range_header)
if mobj:
start = int(mobj.group(1))
end = int(mobj.group(2))
valid_range = start is not None and end is not None
if valid_range:
content_range = 'bytes %d-%d' % (start, end)
if total:
content_range += '/%d' % total
self.send_header('Content-Range', content_range)
return (end - start + 1) if valid_range else total
def serve(self, range=True, content_length=True):
self.send_response(200)
self.send_header('Content-Type', 'video/mp4')
size = TEST_SIZE
if range:
size = self.send_content_range(TEST_SIZE)
if content_length:
self.send_header('Content-Length', size)
self.end_headers()
self.wfile.write(b'#' * size)
def do_GET(self):
if self.path == '/regular':
self.serve()
elif self.path == '/no-content-length':
self.serve(content_length=False)
elif self.path == '/no-range':
self.serve(range=False)
elif self.path == '/no-range-no-content-length':
self.serve(range=False, content_length=False)
else:
assert False
class TestHttpFD(unittest.TestCase):
def setUp(self):
self.httpd = compat_http_server.HTTPServer(
('127.0.0.1', 0), HTTPTestRequestHandler)
self.port = http_server_port(self.httpd)
self.server_thread = threading.Thread(target=self.httpd.serve_forever)
self.server_thread.daemon = True
self.server_thread.start()
def download(self, params, ep):
params['logger'] = FakeLogger()
ydl = YoutubeDL(params)
downloader = HttpFD(ydl, params)
filename = 'testfile.mp4'
try_rm(encodeFilename(filename))
self.assertTrue(downloader.real_download(filename, {
'url': 'http://127.0.0.1:%d/%s' % (self.port, ep),
}))
self.assertEqual(os.path.getsize(encodeFilename(filename)), TEST_SIZE, ep)
try_rm(encodeFilename(filename))
def download_all(self, params):
for ep in ('regular', 'no-content-length', 'no-range', 'no-range-no-content-length'):
self.download(params, ep)
def test_regular(self):
self.download_all({})
def test_chunked(self):
self.download_all({
'http_chunk_size': 1000,
})
if __name__ == '__main__':
unittest.main()

View File

@ -8,36 +8,55 @@ import unittest
import sys
import os
import subprocess
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from youtube_dl.utils import encodeArgument
rootDir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, rootDir)
try:
_DEV_NULL = subprocess.DEVNULL
except AttributeError:
_DEV_NULL = open(os.devnull, 'wb')
from youtube_dl.compat import compat_register_utf8, compat_subprocess_get_DEVNULL
from youtube_dl.utils import encodeArgument
compat_register_utf8()
_DEV_NULL = compat_subprocess_get_DEVNULL()
class TestExecution(unittest.TestCase):
def setUp(self):
self.module = 'youtube_dl'
if sys.version_info < (2, 7):
self.module += '.__main__'
def test_import(self):
subprocess.check_call([sys.executable, '-c', 'import youtube_dl'], cwd=rootDir)
def test_module_exec(self):
if sys.version_info >= (2, 7): # Python 2.6 doesn't support package execution
subprocess.check_call([sys.executable, '-m', 'youtube_dl', '--version'], cwd=rootDir, stdout=_DEV_NULL)
subprocess.check_call([sys.executable, '-m', self.module, '--version'], cwd=rootDir, stdout=_DEV_NULL)
def test_main_exec(self):
subprocess.check_call([sys.executable, 'youtube_dl/__main__.py', '--version'], cwd=rootDir, stdout=_DEV_NULL)
subprocess.check_call([sys.executable, os.path.normpath('youtube_dl/__main__.py'), '--version'], cwd=rootDir, stdout=_DEV_NULL)
def test_cmdline_umlauts(self):
os.environ['PYTHONIOENCODING'] = 'utf-8'
p = subprocess.Popen(
[sys.executable, 'youtube_dl/__main__.py', encodeArgument('ä'), '--version'],
[sys.executable, '-m', self.module, encodeArgument('ä'), '--version'],
cwd=rootDir, stdout=_DEV_NULL, stderr=subprocess.PIPE)
_, stderr = p.communicate()
self.assertFalse(stderr)
def test_lazy_extractors(self):
lazy_extractors = os.path.normpath('youtube_dl/extractor/lazy_extractors.py')
try:
subprocess.check_call([sys.executable, os.path.normpath('devscripts/make_lazy_extractors.py'), lazy_extractors], cwd=rootDir, stdout=_DEV_NULL)
subprocess.check_call([sys.executable, os.path.normpath('test/test_all_urls.py')], cwd=rootDir, stdout=_DEV_NULL)
finally:
for x in ('', 'c') if sys.version_info[0] < 3 else ('',):
try:
os.remove(lazy_extractors + x)
except OSError:
pass
if __name__ == '__main__':
unittest.main()

View File

@ -1,4 +1,5 @@
#!/usr/bin/env python
# coding: utf-8
from __future__ import unicode_literals
# Allow direct execution
@ -7,66 +8,500 @@ import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from youtube_dl import YoutubeDL
from youtube_dl.compat import compat_http_server, compat_urllib_request
import contextlib
import gzip
import io
import ssl
import tempfile
import threading
import zlib
# avoid deprecated alias assertRaisesRegexp
if hasattr(unittest.TestCase, 'assertRaisesRegex'):
unittest.TestCase.assertRaisesRegexp = unittest.TestCase.assertRaisesRegex
try:
import brotli
except ImportError:
brotli = None
try:
from urllib.request import pathname2url
except ImportError:
from urllib import pathname2url
from youtube_dl.compat import (
compat_http_cookiejar_Cookie,
compat_http_server,
compat_str as str,
compat_urllib_error,
compat_urllib_HTTPError,
compat_urllib_parse,
compat_urllib_request,
)
from youtube_dl.utils import (
sanitized_Request,
update_Request,
urlencode_postdata,
)
from test.helper import (
expectedFailureIf,
FakeYDL,
FakeLogger,
http_server_port,
)
from youtube_dl import YoutubeDL
TEST_DIR = os.path.dirname(os.path.abspath(__file__))
class HTTPTestRequestHandler(compat_http_server.BaseHTTPRequestHandler):
protocol_version = 'HTTP/1.1'
# work-around old/new -style class inheritance
def super(self, meth_name, *args, **kwargs):
from types import MethodType
try:
super()
fn = lambda s, m, *a, **k: getattr(super(), m)(*a, **k)
except TypeError:
fn = lambda s, m, *a, **k: getattr(compat_http_server.BaseHTTPRequestHandler, m)(s, *a, **k)
self.super = MethodType(fn, self)
return self.super(meth_name, *args, **kwargs)
def log_message(self, format, *args):
pass
def do_GET(self):
if self.path == '/video.html':
self.send_response(200)
self.send_header('Content-Type', 'text/html; charset=utf-8')
self.end_headers()
self.wfile.write(b'<html><video src="/vid.mp4" /></html>')
elif self.path == '/vid.mp4':
self.send_response(200)
self.send_header('Content-Type', 'video/mp4')
self.end_headers()
self.wfile.write(b'\x00\x00\x00\x00\x20\x66\x74[video]')
def _headers(self):
payload = str(self.headers).encode('utf-8')
self.send_response(200)
self.send_header('Content-Type', 'application/json')
self.send_header('Content-Length', str(len(payload)))
self.end_headers()
self.wfile.write(payload)
def _redirect(self):
self.send_response(int(self.path[len('/redirect_'):]))
self.send_header('Location', '/method')
self.send_header('Content-Length', '0')
self.end_headers()
def _method(self, method, payload=None):
self.send_response(200)
self.send_header('Content-Length', str(len(payload or '')))
self.send_header('Method', method)
self.end_headers()
if payload:
self.wfile.write(payload)
def _status(self, status):
payload = '<html>{0} NOT FOUND</html>'.format(status).encode('utf-8')
self.send_response(int(status))
self.send_header('Content-Type', 'text/html; charset=utf-8')
self.send_header('Content-Length', str(len(payload)))
self.end_headers()
self.wfile.write(payload)
def _read_data(self):
if 'Content-Length' in self.headers:
return self.rfile.read(int(self.headers['Content-Length']))
def _test_url(self, path, host='127.0.0.1', scheme='http', port=None):
return '{0}://{1}:{2}/{3}'.format(
scheme, host,
port if port is not None
else http_server_port(self.server), path)
def do_POST(self):
data = self._read_data()
if self.path.startswith('/redirect_'):
self._redirect()
elif self.path.startswith('/method'):
self._method('POST', data)
elif self.path.startswith('/headers'):
self._headers()
else:
assert False
self._status(404)
def do_HEAD(self):
if self.path.startswith('/redirect_'):
self._redirect()
elif self.path.startswith('/method'):
self._method('HEAD')
else:
self._status(404)
class FakeLogger(object):
def debug(self, msg):
pass
def do_PUT(self):
data = self._read_data()
if self.path.startswith('/redirect_'):
self._redirect()
elif self.path.startswith('/method'):
self._method('PUT', data)
else:
self._status(404)
def warning(self, msg):
pass
def do_GET(self):
def error(self, msg):
pass
def respond(payload=b'<html><video src="/vid.mp4" /></html>',
payload_type='text/html; charset=utf-8',
payload_encoding=None,
resp_code=200):
self.send_response(resp_code)
self.send_header('Content-Type', payload_type)
if payload_encoding:
self.send_header('Content-Encoding', payload_encoding)
self.send_header('Content-Length', str(len(payload))) # required for persistent connections
self.end_headers()
self.wfile.write(payload)
def gzip_compress(p):
buf = io.BytesIO()
with contextlib.closing(gzip.GzipFile(fileobj=buf, mode='wb')) as f:
f.write(p)
return buf.getvalue()
if self.path == '/video.html':
respond()
elif self.path == '/vid.mp4':
respond(b'\x00\x00\x00\x00\x20\x66\x74[video]', 'video/mp4')
elif self.path == '/302':
if sys.version_info[0] == 3:
# XXX: Python 3 http server does not allow non-ASCII header values
self.send_response(404)
self.end_headers()
return
new_url = self._test_url('中文.html')
self.send_response(302)
self.send_header(b'Location', new_url.encode('utf-8'))
self.end_headers()
elif self.path == '/%E4%B8%AD%E6%96%87.html':
respond()
elif self.path == '/%c7%9f':
respond()
elif self.path == '/redirect_dotsegments':
self.send_response(301)
# redirect to /headers but with dot segments before
self.send_header('Location', '/a/b/./../../headers')
self.send_header('Content-Length', '0')
self.end_headers()
elif self.path.startswith('/redirect_'):
self._redirect()
elif self.path.startswith('/method'):
self._method('GET')
elif self.path.startswith('/headers'):
self._headers()
elif self.path.startswith('/308-to-headers'):
self.send_response(308)
self.send_header('Location', '/headers')
self.send_header('Content-Length', '0')
self.end_headers()
elif self.path == '/trailing_garbage':
payload = b'<html><video src="/vid.mp4" /></html>'
compressed = gzip_compress(payload) + b'trailing garbage'
respond(compressed, payload_encoding='gzip')
elif self.path == '/302-non-ascii-redirect':
new_url = self._test_url('中文.html')
# actually respond with permanent redirect
self.send_response(301)
self.send_header('Location', new_url)
self.send_header('Content-Length', '0')
self.end_headers()
elif self.path == '/content-encoding':
encodings = self.headers.get('ytdl-encoding', '')
payload = b'<html><video src="/vid.mp4" /></html>'
for encoding in filter(None, (e.strip() for e in encodings.split(','))):
if encoding == 'br' and brotli:
payload = brotli.compress(payload)
elif encoding == 'gzip':
payload = gzip_compress(payload)
elif encoding == 'deflate':
payload = zlib.compress(payload)
elif encoding == 'unsupported':
payload = b'raw'
break
else:
self._status(415)
return
respond(payload, payload_encoding=encodings)
else:
self._status(404)
def send_header(self, keyword, value):
"""
Forcibly allow HTTP server to send non percent-encoded non-ASCII characters in headers.
This is against what is defined in RFC 3986: but we need to test that we support this
since some sites incorrectly do this.
"""
if keyword.lower() == 'connection':
return self.super('send_header', keyword, value)
if not hasattr(self, '_headers_buffer'):
self._headers_buffer = []
self._headers_buffer.append('{0}: {1}\r\n'.format(keyword, value).encode('utf-8'))
def end_headers(self):
if hasattr(self, '_headers_buffer'):
self.wfile.write(b''.join(self._headers_buffer))
self._headers_buffer = []
self.super('end_headers')
class TestHTTP(unittest.TestCase):
# when does it make sense to check the SSL certificate?
_check_cert = (
sys.version_info >= (3, 2)
or (sys.version_info[0] == 2 and sys.version_info[1:] >= (7, 19)))
def setUp(self):
# HTTP server
self.http_httpd = compat_http_server.HTTPServer(
('127.0.0.1', 0), HTTPTestRequestHandler)
self.http_port = http_server_port(self.http_httpd)
self.http_server_thread = threading.Thread(target=self.http_httpd.serve_forever)
self.http_server_thread.daemon = True
self.http_server_thread.start()
try:
from http.server import ThreadingHTTPServer
except ImportError:
try:
from socketserver import ThreadingMixIn
except ImportError:
from SocketServer import ThreadingMixIn
class ThreadingHTTPServer(ThreadingMixIn, compat_http_server.HTTPServer):
pass
# HTTPS server
certfn = os.path.join(TEST_DIR, 'testcert.pem')
self.httpd = compat_http_server.HTTPServer(
('localhost', 0), HTTPTestRequestHandler)
self.httpd.socket = ssl.wrap_socket(
self.httpd.socket, certfile=certfn, server_side=True)
self.port = self.httpd.socket.getsockname()[1]
self.server_thread = threading.Thread(target=self.httpd.serve_forever)
self.server_thread.daemon = True
self.server_thread.start()
self.https_httpd = ThreadingHTTPServer(
('127.0.0.1', 0), HTTPTestRequestHandler)
try:
sslctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
sslctx.verify_mode = ssl.CERT_NONE
sslctx.check_hostname = False
sslctx.load_cert_chain(certfn, None)
self.https_httpd.socket = sslctx.wrap_socket(
self.https_httpd.socket, server_side=True)
except AttributeError:
self.https_httpd.socket = ssl.wrap_socket(
self.https_httpd.socket, certfile=certfn, server_side=True)
self.https_port = http_server_port(self.https_httpd)
self.https_server_thread = threading.Thread(target=self.https_httpd.serve_forever)
self.https_server_thread.daemon = True
self.https_server_thread.start()
def tearDown(self):
def closer(svr):
def _closer():
svr.shutdown()
svr.server_close()
return _closer
shutdown_thread = threading.Thread(target=closer(self.http_httpd))
shutdown_thread.start()
self.http_server_thread.join(2.0)
shutdown_thread = threading.Thread(target=closer(self.https_httpd))
shutdown_thread.start()
self.https_server_thread.join(2.0)
def _test_url(self, path, host='127.0.0.1', scheme='http', port=None):
return '{0}://{1}:{2}/{3}'.format(
scheme, host,
port if port is not None
else self.https_port if scheme == 'https'
else self.http_port, path)
@unittest.skipUnless(_check_cert, 'No support for certificate check in SSL')
def test_nocheckcertificate(self):
if sys.version_info >= (2, 7, 9): # No certificate checking anyways
ydl = YoutubeDL({'logger': FakeLogger()})
self.assertRaises(
Exception,
ydl.extract_info, 'https://localhost:%d/video.html' % self.port)
with FakeYDL({'logger': FakeLogger()}) as ydl:
with self.assertRaises(compat_urllib_error.URLError):
ydl.urlopen(sanitized_Request(self._test_url('headers', scheme='https')))
ydl = YoutubeDL({'logger': FakeLogger(), 'nocheckcertificate': True})
r = ydl.extract_info('https://localhost:%d/video.html' % self.port)
self.assertEqual(r['url'], 'https://localhost:%d/vid.mp4' % self.port)
with FakeYDL({'logger': FakeLogger(), 'nocheckcertificate': True}) as ydl:
r = ydl.urlopen(sanitized_Request(self._test_url('headers', scheme='https')))
self.assertEqual(r.getcode(), 200)
r.close()
def test_percent_encode(self):
with FakeYDL() as ydl:
# Unicode characters should be encoded with uppercase percent-encoding
res = ydl.urlopen(sanitized_Request(self._test_url('中文.html')))
self.assertEqual(res.getcode(), 200)
res.close()
# don't normalize existing percent encodings
res = ydl.urlopen(sanitized_Request(self._test_url('%c7%9f')))
self.assertEqual(res.getcode(), 200)
res.close()
def test_unicode_path_redirection(self):
with FakeYDL() as ydl:
r = ydl.urlopen(sanitized_Request(self._test_url('302-non-ascii-redirect')))
self.assertEqual(r.url, self._test_url('%E4%B8%AD%E6%96%87.html'))
r.close()
def test_redirect(self):
with FakeYDL() as ydl:
def do_req(redirect_status, method, check_no_content=False):
data = b'testdata' if method in ('POST', 'PUT') else None
res = ydl.urlopen(sanitized_Request(
self._test_url('redirect_{0}'.format(redirect_status)),
method=method, data=data))
if check_no_content:
self.assertNotIn('Content-Type', res.headers)
return res.read().decode('utf-8'), res.headers.get('method', '')
# A 303 must either use GET or HEAD for subsequent request
self.assertEqual(do_req(303, 'POST'), ('', 'GET'))
self.assertEqual(do_req(303, 'HEAD'), ('', 'HEAD'))
self.assertEqual(do_req(303, 'PUT'), ('', 'GET'))
# 301 and 302 turn POST only into a GET, with no Content-Type
self.assertEqual(do_req(301, 'POST', True), ('', 'GET'))
self.assertEqual(do_req(301, 'HEAD'), ('', 'HEAD'))
self.assertEqual(do_req(302, 'POST', True), ('', 'GET'))
self.assertEqual(do_req(302, 'HEAD'), ('', 'HEAD'))
self.assertEqual(do_req(301, 'PUT'), ('testdata', 'PUT'))
self.assertEqual(do_req(302, 'PUT'), ('testdata', 'PUT'))
# 307 and 308 should not change method
for m in ('POST', 'PUT'):
self.assertEqual(do_req(307, m), ('testdata', m))
self.assertEqual(do_req(308, m), ('testdata', m))
self.assertEqual(do_req(307, 'HEAD'), ('', 'HEAD'))
self.assertEqual(do_req(308, 'HEAD'), ('', 'HEAD'))
# These should not redirect and instead raise an HTTPError
for code in (300, 304, 305, 306):
with self.assertRaises(compat_urllib_HTTPError):
do_req(code, 'GET')
# Jython 2.7.1 times out for some reason
@expectedFailureIf(sys.platform.startswith('java') and sys.version_info < (2, 7, 2))
def test_content_type(self):
# https://github.com/yt-dlp/yt-dlp/commit/379a4f161d4ad3e40932dcf5aca6e6fb9715ab28
with FakeYDL({'nocheckcertificate': True}) as ydl:
# method should be auto-detected as POST
r = sanitized_Request(self._test_url('headers', scheme='https'), data=urlencode_postdata({'test': 'test'}))
headers = ydl.urlopen(r).read().decode('utf-8')
self.assertIn('Content-Type: application/x-www-form-urlencoded', headers)
# test http
r = sanitized_Request(self._test_url('headers'), data=urlencode_postdata({'test': 'test'}))
headers = ydl.urlopen(r).read().decode('utf-8')
self.assertIn('Content-Type: application/x-www-form-urlencoded', headers)
def test_update_req(self):
req = sanitized_Request('http://example.com')
assert req.data is None
assert req.get_method() == 'GET'
assert not req.has_header('Content-Type')
# Test that zero-byte payloads will be sent
req = update_Request(req, data=b'')
assert req.data == b''
assert req.get_method() == 'POST'
# yt-dl expects data to be encoded and Content-Type to be added by sender
# assert req.get_header('Content-Type') == 'application/x-www-form-urlencoded'
def test_cookiejar(self):
with FakeYDL() as ydl:
ydl.cookiejar.set_cookie(compat_http_cookiejar_Cookie(
0, 'test', 'ytdl', None, False, '127.0.0.1', True,
False, '/headers', True, False, None, False, None, None, {}))
data = ydl.urlopen(sanitized_Request(
self._test_url('headers'))).read().decode('utf-8')
self.assertIn('Cookie: test=ytdl', data)
def test_passed_cookie_header(self):
# We should accept a Cookie header being passed as in normal headers and handle it appropriately.
with FakeYDL() as ydl:
# Specified Cookie header should be used
res = ydl.urlopen(sanitized_Request(
self._test_url('headers'), headers={'Cookie': 'test=test'})).read().decode('utf-8')
self.assertIn('Cookie: test=test', res)
# Specified Cookie header should be removed on any redirect
res = ydl.urlopen(sanitized_Request(
self._test_url('308-to-headers'), headers={'Cookie': 'test=test'})).read().decode('utf-8')
self.assertNotIn('Cookie: test=test', res)
# Specified Cookie header should override global cookiejar for that request
ydl.cookiejar.set_cookie(compat_http_cookiejar_Cookie(
0, 'test', 'ytdlp', None, False, '127.0.0.1', True,
False, '/headers', True, False, None, False, None, None, {}))
data = ydl.urlopen(sanitized_Request(
self._test_url('headers'), headers={'Cookie': 'test=test'})).read().decode('utf-8')
self.assertNotIn('Cookie: test=ytdlp', data)
self.assertIn('Cookie: test=test', data)
def test_no_compression_compat_header(self):
with FakeYDL() as ydl:
data = ydl.urlopen(
sanitized_Request(
self._test_url('headers'),
headers={'Youtubedl-no-compression': True})).read()
self.assertIn(b'Accept-Encoding: identity', data)
self.assertNotIn(b'youtubedl-no-compression', data.lower())
def test_gzip_trailing_garbage(self):
# https://github.com/ytdl-org/youtube-dl/commit/aa3e950764337ef9800c936f4de89b31c00dfcf5
# https://github.com/ytdl-org/youtube-dl/commit/6f2ec15cee79d35dba065677cad9da7491ec6e6f
with FakeYDL() as ydl:
data = ydl.urlopen(sanitized_Request(self._test_url('trailing_garbage'))).read().decode('utf-8')
self.assertEqual(data, '<html><video src="/vid.mp4" /></html>')
def __test_compression(self, encoding):
with FakeYDL() as ydl:
res = ydl.urlopen(
sanitized_Request(
self._test_url('content-encoding'),
headers={'ytdl-encoding': encoding}))
# decoded encodings are removed: only check for valid decompressed data
self.assertEqual(res.read(), b'<html><video src="/vid.mp4" /></html>')
@unittest.skipUnless(brotli, 'brotli support is not installed')
def test_brotli(self):
self.__test_compression('br')
def test_deflate(self):
self.__test_compression('deflate')
def test_gzip(self):
self.__test_compression('gzip')
def test_multiple_encodings(self):
# https://www.rfc-editor.org/rfc/rfc9110.html#section-8.4
for pair in ('gzip,deflate', 'deflate, gzip', 'gzip, gzip', 'deflate, deflate'):
self.__test_compression(pair)
def test_unsupported_encoding(self):
# it should return the raw content
with FakeYDL() as ydl:
res = ydl.urlopen(
sanitized_Request(
self._test_url('content-encoding'),
headers={'ytdl-encoding': 'unsupported'}))
self.assertEqual(res.headers.get('Content-Encoding'), 'unsupported')
self.assertEqual(res.read(), b'raw')
def test_remove_dot_segments(self):
with FakeYDL() as ydl:
res = ydl.urlopen(sanitized_Request(self._test_url('a/b/./../../headers')))
self.assertEqual(compat_urllib_parse.urlparse(res.geturl()).path, '/headers')
res = ydl.urlopen(sanitized_Request(self._test_url('redirect_dotsegments')))
self.assertEqual(compat_urllib_parse.urlparse(res.geturl()).path, '/headers')
def _build_proxy_handler(name):
@ -80,40 +515,90 @@ def _build_proxy_handler(name):
self.send_response(200)
self.send_header('Content-Type', 'text/plain; charset=utf-8')
self.end_headers()
self.wfile.write('{self.proxy_name}: {self.path}'.format(self=self).encode('utf-8'))
self.wfile.write('{0}: {1}'.format(self.proxy_name, self.path).encode('utf-8'))
return HTTPTestRequestHandler
class TestProxy(unittest.TestCase):
def setUp(self):
self.proxy = compat_http_server.HTTPServer(
('localhost', 0), _build_proxy_handler('normal'))
self.port = self.proxy.socket.getsockname()[1]
('127.0.0.1', 0), _build_proxy_handler('normal'))
self.port = http_server_port(self.proxy)
self.proxy_thread = threading.Thread(target=self.proxy.serve_forever)
self.proxy_thread.daemon = True
self.proxy_thread.start()
self.cn_proxy = compat_http_server.HTTPServer(
('localhost', 0), _build_proxy_handler('cn'))
self.cn_port = self.cn_proxy.socket.getsockname()[1]
self.cn_proxy_thread = threading.Thread(target=self.cn_proxy.serve_forever)
self.cn_proxy_thread.daemon = True
self.cn_proxy_thread.start()
self.geo_proxy = compat_http_server.HTTPServer(
('127.0.0.1', 0), _build_proxy_handler('geo'))
self.geo_port = http_server_port(self.geo_proxy)
self.geo_proxy_thread = threading.Thread(target=self.geo_proxy.serve_forever)
self.geo_proxy_thread.daemon = True
self.geo_proxy_thread.start()
def tearDown(self):
def closer(svr):
def _closer():
svr.shutdown()
svr.server_close()
return _closer
shutdown_thread = threading.Thread(target=closer(self.proxy))
shutdown_thread.start()
self.proxy_thread.join(2.0)
shutdown_thread = threading.Thread(target=closer(self.geo_proxy))
shutdown_thread.start()
self.geo_proxy_thread.join(2.0)
def _test_proxy(self, host='127.0.0.1', port=None):
return '{0}:{1}'.format(
host, port if port is not None else self.port)
def test_proxy(self):
cn_proxy = 'localhost:{0}'.format(self.cn_port)
geo_proxy = self._test_proxy(port=self.geo_port)
ydl = YoutubeDL({
'proxy': 'localhost:{0}'.format(self.port),
'cn_verification_proxy': cn_proxy,
'proxy': self._test_proxy(),
'geo_verification_proxy': geo_proxy,
})
url = 'http://foo.com/bar'
response = ydl.urlopen(url).read().decode('utf-8')
self.assertEqual(response, 'normal: {0}'.format(url))
req = compat_urllib_request.Request(url)
req.add_header('Ytdl-request-proxy', cn_proxy)
req.add_header('Ytdl-request-proxy', geo_proxy)
response = ydl.urlopen(req).read().decode('utf-8')
self.assertEqual(response, 'cn: {0}'.format(url))
self.assertEqual(response, 'geo: {0}'.format(url))
def test_proxy_with_idn(self):
ydl = YoutubeDL({
'proxy': self._test_proxy(),
})
url = 'http://中文.tw/'
response = ydl.urlopen(url).read().decode('utf-8')
# b'xn--fiq228c' is '中文'.encode('idna')
self.assertEqual(response, 'normal: http://xn--fiq228c.tw/')
class TestFileURL(unittest.TestCase):
# See https://github.com/ytdl-org/youtube-dl/issues/8227
def test_file_urls(self):
tf = tempfile.NamedTemporaryFile(delete=False)
tf.write(b'foobar')
tf.close()
url = compat_urllib_parse.urljoin('file://', pathname2url(tf.name))
with FakeYDL() as ydl:
self.assertRaisesRegexp(
compat_urllib_error.URLError, 'file:// scheme is explicitly disabled in youtube-dl for security reasons', ydl.urlopen, url)
# not yet implemented
"""
with FakeYDL({'enable_file_urls': True}) as ydl:
res = ydl.urlopen(url)
self.assertEqual(res.read(), b'foobar')
res.close()
"""
os.unlink(tf.name)
if __name__ == '__main__':
unittest.main()

View File

@ -0,0 +1,48 @@
#!/usr/bin/env python
from __future__ import unicode_literals
# Allow direct execution
import os
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from test.helper import FakeYDL
from youtube_dl.extractor import IqiyiIE
class IqiyiIEWithCredentials(IqiyiIE):
def _get_login_info(self):
return 'foo', 'bar'
class WarningLogger(object):
def __init__(self):
self.messages = []
def warning(self, msg):
self.messages.append(msg)
def debug(self, msg):
pass
def error(self, msg):
pass
class TestIqiyiSDKInterpreter(unittest.TestCase):
def test_iqiyi_sdk_interpreter(self):
'''
Test the functionality of IqiyiSDKInterpreter by trying to log in
If `sign` is incorrect, /validate call throws an HTTP 556 error
'''
logger = WarningLogger()
ie = IqiyiIEWithCredentials(FakeYDL({'logger': logger}))
ie._login()
self.assertTrue('unable to log in:' in logger.messages[0])
if __name__ == '__main__':
unittest.main()

View File

@ -1,4 +1,5 @@
#!/usr/bin/env python
# coding: utf-8
from __future__ import unicode_literals
@ -6,103 +7,650 @@ from __future__ import unicode_literals
import os
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from youtube_dl.jsinterp import JSInterpreter
import math
import re
import time
from youtube_dl.compat import compat_str as str
from youtube_dl.jsinterp import JS_Undefined, JSInterpreter
NaN = object()
class TestJSInterpreter(unittest.TestCase):
def _test(self, jsi_or_code, expected, func='f', args=()):
if isinstance(jsi_or_code, str):
jsi_or_code = JSInterpreter(jsi_or_code)
got = jsi_or_code.call_function(func, *args)
if expected is NaN:
self.assertTrue(math.isnan(got), '{0} is not NaN'.format(got))
else:
self.assertEqual(got, expected)
def test_basic(self):
jsi = JSInterpreter('function x(){;}')
self.assertEqual(jsi.call_function('x'), None)
jsi = JSInterpreter('function f(){;}')
self.assertEqual(repr(jsi.extract_function('f')), 'F<f>')
self._test(jsi, None)
jsi = JSInterpreter('function x3(){return 42;}')
self.assertEqual(jsi.call_function('x3'), 42)
self._test('function f(){return 42;}', 42)
self._test('function f(){42}', None)
self._test('var f = function(){return 42;}', 42)
jsi = JSInterpreter('var x5 = function(){return 42;}')
self.assertEqual(jsi.call_function('x5'), 42)
def test_add(self):
self._test('function f(){return 42 + 7;}', 49)
self._test('function f(){return 42 + undefined;}', NaN)
self._test('function f(){return 42 + null;}', 42)
self._test('function f(){return 1 + "";}', '1')
self._test('function f(){return 42 + "7";}', '427')
self._test('function f(){return false + true;}', 1)
self._test('function f(){return "false" + true;}', 'falsetrue')
self._test('function f(){return '
'1 + "2" + [3,4] + {k: 56} + null + undefined + Infinity;}',
'123,4[object Object]nullundefinedInfinity')
def test_sub(self):
self._test('function f(){return 42 - 7;}', 35)
self._test('function f(){return 42 - undefined;}', NaN)
self._test('function f(){return 42 - null;}', 42)
self._test('function f(){return 42 - "7";}', 35)
self._test('function f(){return 42 - "spam";}', NaN)
def test_mul(self):
self._test('function f(){return 42 * 7;}', 294)
self._test('function f(){return 42 * undefined;}', NaN)
self._test('function f(){return 42 * null;}', 0)
self._test('function f(){return 42 * "7";}', 294)
self._test('function f(){return 42 * "eggs";}', NaN)
def test_div(self):
jsi = JSInterpreter('function f(a, b){return a / b;}')
self._test(jsi, NaN, args=(0, 0))
self._test(jsi, NaN, args=(JS_Undefined, 1))
self._test(jsi, float('inf'), args=(2, 0))
self._test(jsi, 0, args=(0, 3))
self._test(jsi, 6, args=(42, 7))
self._test(jsi, 0, args=(42, float('inf')))
self._test(jsi, 6, args=("42", 7))
self._test(jsi, NaN, args=("spam", 7))
def test_mod(self):
self._test('function f(){return 42 % 7;}', 0)
self._test('function f(){return 42 % 0;}', NaN)
self._test('function f(){return 42 % undefined;}', NaN)
self._test('function f(){return 42 % "7";}', 0)
self._test('function f(){return 42 % "beans";}', NaN)
def test_exp(self):
self._test('function f(){return 42 ** 2;}', 1764)
self._test('function f(){return 42 ** undefined;}', NaN)
self._test('function f(){return 42 ** null;}', 1)
self._test('function f(){return undefined ** 0;}', 1)
self._test('function f(){return undefined ** 42;}', NaN)
self._test('function f(){return 42 ** "2";}', 1764)
self._test('function f(){return 42 ** "spam";}', NaN)
def test_calc(self):
jsi = JSInterpreter('function x4(a){return 2*a+1;}')
self.assertEqual(jsi.call_function('x4', 3), 7)
self._test('function f(a){return 2*a+1;}', 7, args=[3])
def test_empty_return(self):
jsi = JSInterpreter('function f(){return; y()}')
self.assertEqual(jsi.call_function('f'), None)
self._test('function f(){return; y()}', None)
def test_morespace(self):
jsi = JSInterpreter('function x (a) { return 2 * a + 1 ; }')
self.assertEqual(jsi.call_function('x', 3), 7)
jsi = JSInterpreter('function f () { x = 2 ; return x; }')
self.assertEqual(jsi.call_function('f'), 2)
self._test('function f (a) { return 2 * a + 1 ; }', 7, args=[3])
self._test('function f () { x = 2 ; return x; }', 2)
def test_strange_chars(self):
jsi = JSInterpreter('function $_xY1 ($_axY1) { var $_axY2 = $_axY1 + 1; return $_axY2; }')
self.assertEqual(jsi.call_function('$_xY1', 20), 21)
self._test('function $_xY1 ($_axY1) { var $_axY2 = $_axY1 + 1; return $_axY2; }',
21, args=[20], func='$_xY1')
def test_operators(self):
jsi = JSInterpreter('function f(){return 1 << 5;}')
self.assertEqual(jsi.call_function('f'), 32)
self._test('function f(){return 1 << 5;}', 32)
self._test('function f(){return 2 ** 5}', 32)
self._test('function f(){return 19 & 21;}', 17)
self._test('function f(){return 11 >> 2;}', 2)
self._test('function f(){return []? 2+3: 4;}', 5)
# equality
self._test('function f(){return 1 == 1}', True)
self._test('function f(){return 1 == 1.0}', True)
self._test('function f(){return 1 == "1"}', True)
self._test('function f(){return 1 == 2}', False)
self._test('function f(){return 1 != "1"}', False)
self._test('function f(){return 1 != 2}', True)
self._test('function f(){var x = {a: 1}; var y = x; return x == y}', True)
self._test('function f(){var x = {a: 1}; return x == {a: 1}}', False)
self._test('function f(){return NaN == NaN}', False)
self._test('function f(){return null == undefined}', True)
self._test('function f(){return "spam, eggs" == "spam, eggs"}', True)
# strict equality
self._test('function f(){return 1 === 1}', True)
self._test('function f(){return 1 === 1.0}', True)
self._test('function f(){return 1 === "1"}', False)
self._test('function f(){return 1 === 2}', False)
self._test('function f(){var x = {a: 1}; var y = x; return x === y}', True)
self._test('function f(){var x = {a: 1}; return x === {a: 1}}', False)
self._test('function f(){return NaN === NaN}', False)
self._test('function f(){return null === undefined}', False)
self._test('function f(){return null === null}', True)
self._test('function f(){return undefined === undefined}', True)
self._test('function f(){return "uninterned" === "uninterned"}', True)
self._test('function f(){return 1 === 1}', True)
self._test('function f(){return 1 === "1"}', False)
self._test('function f(){return 1 !== 1}', False)
self._test('function f(){return 1 !== "1"}', True)
# expressions
self._test('function f(){return 0 && 1 || 2;}', 2)
self._test('function f(){return 0 ?? 42;}', 0)
self._test('function f(){return "life, the universe and everything" < 42;}', False)
# https://github.com/ytdl-org/youtube-dl/issues/32815
self._test('function f(){return 0 - 7 * - 6;}', 42)
jsi = JSInterpreter('function f(){return 19 & 21;}')
self.assertEqual(jsi.call_function('f'), 17)
jsi = JSInterpreter('function f(){return 11 >> 2;}')
self.assertEqual(jsi.call_function('f'), 2)
def test_bitwise_operators_typecast(self):
# madness
self._test('function f(){return null << 5}', 0)
self._test('function f(){return undefined >> 5}', 0)
self._test('function f(){return 42 << NaN}', 42)
self._test('function f(){return 42 << Infinity}', 42)
self._test('function f(){return 0.0 << null}', 0)
self._test('function f(){return NaN << 42}', 0)
self._test('function f(){return "21.9" << 1}', 42)
self._test('function f(){return true << "5";}', 32)
self._test('function f(){return true << true;}', 2)
self._test('function f(){return "19" & "21.9";}', 17)
self._test('function f(){return "19" & false;}', 0)
self._test('function f(){return "11.0" >> "2.1";}', 2)
self._test('function f(){return 5 ^ 9;}', 12)
self._test('function f(){return 0.0 << NaN}', 0)
self._test('function f(){return null << undefined}', 0)
self._test('function f(){return 21 << 4294967297}', 42)
def test_array_access(self):
jsi = JSInterpreter('function f(){var x = [1,2,3]; x[0] = 4; x[0] = 5; x[2] = 7; return x;}')
self.assertEqual(jsi.call_function('f'), [5, 2, 7])
self._test('function f(){var x = [1,2,3]; x[0] = 4; x[0] = 5; x[2.0] = 7; return x;}', [5, 2, 7])
def test_parens(self):
jsi = JSInterpreter('function f(){return (1) + (2) * ((( (( (((((3)))))) )) ));}')
self.assertEqual(jsi.call_function('f'), 7)
self._test('function f(){return (1) + (2) * ((( (( (((((3)))))) )) ));}', 7)
self._test('function f(){return (1 + 2) * 3;}', 9)
jsi = JSInterpreter('function f(){return (1 + 2) * 3;}')
self.assertEqual(jsi.call_function('f'), 9)
def test_quotes(self):
self._test(r'function f(){return "a\"\\("}', r'a"\(')
def test_assignments(self):
jsi = JSInterpreter('function f(){var x = 20; x = 30 + 1; return x;}')
self.assertEqual(jsi.call_function('f'), 31)
jsi = JSInterpreter('function f(){var x = 20; x += 30 + 1; return x;}')
self.assertEqual(jsi.call_function('f'), 51)
jsi = JSInterpreter('function f(){var x = 20; x -= 30 + 1; return x;}')
self.assertEqual(jsi.call_function('f'), -11)
self._test('function f(){var x = 20; x = 30 + 1; return x;}', 31)
self._test('function f(){var x = 20; x += 30 + 1; return x;}', 51)
self._test('function f(){var x = 20; x -= 30 + 1; return x;}', -11)
self._test('function f(){var x = 2; var y = ["a", "b"]; y[x%y["length"]]="z"; return y}', ['z', 'b'])
def test_comments(self):
'Skipping: Not yet fully implemented'
return
jsi = JSInterpreter('''
function x() {
var x = /* 1 + */ 2;
var y = /* 30
* 40 */ 50;
return x + y;
}
''')
self.assertEqual(jsi.call_function('x'), 52)
self._test('''
function f() {
var x = /* 1 + */ 2;
var y = /* 30
* 40 */ 50;
return x + y;
}
''', 52)
jsi = JSInterpreter('''
function f() {
var x = "/*";
var y = 1 /* comment */ + 2;
return y;
}
''')
self.assertEqual(jsi.call_function('f'), 3)
self._test('''
function f() {
var x = "/*";
var y = 1 /* comment */ + 2;
return y;
}
''', 3)
self._test('''
function f() {
var x = ( /* 1 + */ 2 +
/* 30 * 40 */
50);
return x;
}
''', 52)
def test_precedence(self):
self._test('''
function f() {
var a = [10, 20, 30, 40, 50];
var b = 6;
a[0]=a[b%a.length];
return a;
}
''', [20, 20, 30, 40, 50])
def test_builtins(self):
self._test('function f() { return NaN }', NaN)
def test_Date(self):
self._test('function f() { return new Date("Wednesday 31 December 1969 18:01:26 MDT") - 0; }', 86000)
jsi = JSInterpreter('function f(dt) { return new Date(dt) - 0; }')
# date format m/d/y
self._test(jsi, 86000, args=['12/31/1969 18:01:26 MDT'])
# epoch 0
self._test(jsi, 0, args=['1 January 1970 00:00:00 UTC'])
# undefined
self._test(jsi, NaN, args=[JS_Undefined])
# y,m,d, ... - may fail with older dates lacking DST data
jsi = JSInterpreter(
'function f() { return new Date(%s); }'
% ('2024, 5, 29, 2, 52, 12, 42',))
self._test(jsi, (
1719625932042 # UK value
+ (
+ 3600 # back to GMT
+ (time.altzone if time.daylight # host's DST
else time.timezone)
) * 1000))
# no arg
self.assertAlmostEqual(JSInterpreter(
'function f() { return new Date() - 0; }').call_function('f'),
time.time() * 1000, delta=100)
# Date.now()
self.assertAlmostEqual(JSInterpreter(
'function f() { return Date.now(); }').call_function('f'),
time.time() * 1000, delta=100)
# Date.parse()
jsi = JSInterpreter('function f(dt) { return Date.parse(dt); }')
self._test(jsi, 0, args=['1 January 1970 00:00:00 UTC'])
# Date.UTC()
jsi = JSInterpreter('function f() { return Date.UTC(%s); }'
% ('1970, 0, 1, 0, 0, 0, 0',))
self._test(jsi, 0)
def test_call(self):
jsi = JSInterpreter('''
function x() {
var a = [10, 20, 30, 40, 50];
var b = 6;
a[0]=a[b%a.length];
return a;
}''')
self.assertEqual(jsi.call_function('x'), [20, 20, 30, 40, 50])
function x() { return 2; }
function y(a) { return x() + (a?a:0); }
function z() { return y(3); }
''')
self._test(jsi, 5, func='z')
self._test(jsi, 2, func='y')
def test_if(self):
self._test('''
function f() {
let a = 9;
if (0==0) {a++}
return a
}
''', 10)
self._test('''
function f() {
if (0==0) {return 10}
}
''', 10)
self._test('''
function f() {
if (0!=0) {return 1}
else {return 10}
}
''', 10)
def test_elseif(self):
self._test('''
function f() {
if (0!=0) {return 1}
else if (1==0) {return 2}
else {return 10}
}
''', 10)
def test_for_loop(self):
self._test('function f() { a=0; for (i=0; i-10; i++) {a++} return a }', 10)
def test_while_loop(self):
self._test('function f() { a=0; while (a<10) {a++} return a }', 10)
def test_switch(self):
jsi = JSInterpreter('''
function f(x) { switch(x){
case 1:x+=1;
case 2:x+=2;
case 3:x+=3;break;
case 4:x+=4;
default:x=0;
} return x }
''')
self._test(jsi, 7, args=[1])
self._test(jsi, 6, args=[3])
self._test(jsi, 0, args=[5])
def test_switch_default(self):
jsi = JSInterpreter('''
function f(x) { switch(x){
case 2: x+=2;
default: x-=1;
case 5:
case 6: x+=6;
case 0: break;
case 1: x+=1;
} return x }
''')
self._test(jsi, 2, args=[1])
self._test(jsi, 11, args=[5])
self._test(jsi, 14, args=[9])
def test_try(self):
self._test('function f() { try{return 10} catch(e){return 5} }', 10)
def test_catch(self):
self._test('function f() { try{throw 10} catch(e){return 5} }', 5)
def test_finally(self):
self._test('function f() { try{throw 10} finally {return 42} }', 42)
self._test('function f() { try{throw 10} catch(e){return 5} finally {return 42} }', 42)
def test_nested_try(self):
self._test('''
function f() {try {
try{throw 10} finally {throw 42}
} catch(e){return 5} }
''', 5)
def test_for_loop_continue(self):
self._test('function f() { a=0; for (i=0; i-10; i++) { continue; a++ } return a }', 0)
def test_for_loop_break(self):
self._test('function f() { a=0; for (i=0; i-10; i++) { break; a++ } return a }', 0)
def test_for_loop_try(self):
self._test('''
function f() {
for (i=0; i-10; i++) { try { if (i == 5) throw i} catch {return 10} finally {break} };
return 42 }
''', 42)
def test_literal_list(self):
self._test('function f() { return [1, 2, "asdf", [5, 6, 7]][3] }', [5, 6, 7])
def test_comma(self):
self._test('function f() { a=5; a -= 1, a+=3; return a }', 7)
self._test('function f() { a=5; return (a -= 1, a+=3, a); }', 7)
self._test('function f() { return (l=[0,1,2,3], function(a, b){return a+b})((l[1], l[2]), l[3]) }', 5)
def test_not(self):
self._test('function f() { return ! undefined; }', True)
self._test('function f() { return !0; }', True)
self._test('function f() { return !!0; }', False)
self._test('function f() { return ![]; }', False)
self._test('function f() { return !0 !== false; }', True)
def test_void(self):
self._test('function f() { return void 42; }', JS_Undefined)
def test_typeof(self):
self._test('function f() { return typeof undefined; }', 'undefined')
self._test('function f() { return typeof NaN; }', 'number')
self._test('function f() { return typeof Infinity; }', 'number')
self._test('function f() { return typeof true; }', 'boolean')
self._test('function f() { return typeof null; }', 'object')
self._test('function f() { return typeof "a string"; }', 'string')
self._test('function f() { return typeof 42; }', 'number')
self._test('function f() { return typeof 42.42; }', 'number')
self._test('function f() { var g = function(){}; return typeof g; }', 'function')
self._test('function f() { return typeof {key: "value"}; }', 'object')
# not yet implemented: Symbol, BigInt
def test_return_function(self):
jsi = JSInterpreter('''
function x() { return [1, function(){return 1}][1] }
''')
self.assertEqual(jsi.call_function('x')([]), 1)
def test_null(self):
self._test('function f() { return null; }', None)
self._test('function f() { return [null > 0, null < 0, null == 0, null === 0]; }',
[False, False, False, False])
self._test('function f() { return [null >= 0, null <= 0]; }', [True, True])
def test_undefined(self):
self._test('function f() { return undefined === undefined; }', True)
self._test('function f() { return undefined; }', JS_Undefined)
self._test('function f() { return undefined ?? 42; }', 42)
self._test('function f() { let v; return v; }', JS_Undefined)
self._test('function f() { let v; return v**0; }', 1)
self._test('function f() { let v; return [v>42, v<=42, v&&42, 42&&v]; }',
[False, False, JS_Undefined, JS_Undefined])
self._test('''
function f() { return [
undefined === undefined,
undefined == undefined,
undefined == null
]; }
''', [True] * 3)
self._test('''
function f() { return [
undefined < undefined,
undefined > undefined,
undefined === 0,
undefined == 0,
undefined < 0,
undefined > 0,
undefined >= 0,
undefined <= 0,
undefined > null,
undefined < null,
undefined === null
]; }
''', [False] * 11)
jsi = JSInterpreter('''
function x() { let v; return [42+v, v+42, v**42, 42**v, 0**v]; }
''')
for y in jsi.call_function('x'):
self.assertTrue(math.isnan(y))
def test_object(self):
self._test('function f() { return {}; }', {})
self._test('function f() { let a = {m1: 42, m2: 0 }; return [a["m1"], a.m2]; }', [42, 0])
self._test('function f() { let a; return a?.qq; }', JS_Undefined)
self._test('function f() { let a = {m1: 42, m2: 0 }; return a?.qq; }', JS_Undefined)
def test_indexing(self):
self._test('function f() { return [1, 2, 3, 4][3]}', 4)
self._test('function f() { return [1, [2, [3, [4]]]][1][1][1][0]}', 4)
self._test('function f() { var o = {1: 2, 3: 4}; return o[3]}', 4)
self._test('function f() { var o = {1: 2, 3: 4}; return o["3"]}', 4)
self._test('function f() { return [1, [2, {3: [4]}]][1][1]["3"][0]}', 4)
self._test('function f() { return [1, 2, 3, 4].length}', 4)
self._test('function f() { var o = {1: 2, 3: 4}; return o.length}', JS_Undefined)
self._test('function f() { var o = {1: 2, 3: 4}; o["length"] = 42; return o.length}', 42)
def test_regex(self):
self._test('function f() { let a=/,,[/,913,/](,)}/; }', None)
self._test('function f() { let a=/,,[/,913,/](,)}/; return a.source; }', ',,[/,913,/](,)}')
jsi = JSInterpreter('''
function x() { let a=/,,[/,913,/](,)}/; "".replace(a, ""); return a; }
''')
attrs = set(('findall', 'finditer', 'match', 'scanner', 'search',
'split', 'sub', 'subn'))
if sys.version_info >= (2, 7):
# documented for 2.6 but may not be found
attrs.update(('flags', 'groupindex', 'groups', 'pattern'))
self.assertSetEqual(set(dir(jsi.call_function('x'))) & attrs, attrs)
jsi = JSInterpreter('''
function x() { let a=/,,[/,913,/](,)}/i; return a; }
''')
self.assertEqual(jsi.call_function('x').flags & ~re.U, re.I)
jsi = JSInterpreter(r'function f() { let a=/,][}",],()}(\[)/; return a; }')
self.assertEqual(jsi.call_function('f').pattern, r',][}",],()}(\[)')
jsi = JSInterpreter(r'function f() { let a=[/[)\\]/]; return a[0]; }')
self.assertEqual(jsi.call_function('f').pattern, r'[)\\]')
def test_replace(self):
self._test('function f() { let a="data-name".replace("data-", ""); return a }',
'name')
self._test('function f() { let a="data-name".replace(new RegExp("^.+-"), ""); return a; }',
'name')
self._test('function f() { let a="data-name".replace(/^.+-/, ""); return a; }',
'name')
self._test('function f() { let a="data-name".replace(/a/g, "o"); return a; }',
'doto-nome')
self._test('function f() { let a="data-name".replaceAll("a", "o"); return a; }',
'doto-nome')
def test_char_code_at(self):
jsi = JSInterpreter('function f(i){return "test".charCodeAt(i)}')
self._test(jsi, 116, args=[0])
self._test(jsi, 101, args=[1])
self._test(jsi, 115, args=[2])
self._test(jsi, 116, args=[3])
self._test(jsi, None, args=[4])
self._test(jsi, 116, args=['not_a_number'])
def test_bitwise_operators_overflow(self):
self._test('function f(){return -524999584 << 5}', 379882496)
self._test('function f(){return 1236566549 << 5}', 915423904)
def test_negative(self):
self._test('function f(){return 2 * -2.0 ;}', -4)
self._test('function f(){return 2 - - -2 ;}', 0)
self._test('function f(){return 2 - - - -2 ;}', 4)
self._test('function f(){return 2 - + + - -2;}', 0)
self._test('function f(){return 2 + - + - -2;}', 0)
def test_32066(self):
self._test(
"function f(){return Math.pow(3, 5) + new Date('1970-01-01T08:01:42.000+08:00') / 1000 * -239 - -24205;}",
70)
@unittest.skip('Not yet working')
def test_packed(self):
self._test(
'''function f(p,a,c,k,e,d){while(c--)if(k[c])p=p.replace(new RegExp('\\b'+c.toString(a)+'\\b','g'),k[c]);return p}''',
'''h 7=g("1j");7.7h({7g:[{33:"w://7f-7e-7d-7c.v.7b/7a/79/78/77/76.74?t=73&s=2s&e=72&f=2t&71=70.0.0.1&6z=6y&6x=6w"}],6v:"w://32.v.u/6u.31",16:"r%",15:"r%",6t:"6s",6r:"",6q:"l",6p:"l",6o:"6n",6m:\'6l\',6k:"6j",9:[{33:"/2u?b=6i&n=50&6h=w://32.v.u/6g.31",6f:"6e"}],1y:{6d:1,6c:\'#6b\',6a:\'#69\',68:"67",66:30,65:r,},"64":{63:"%62 2m%m%61%5z%5y%5x.u%5w%5v%5u.2y%22 2k%m%1o%22 5t%m%1o%22 5s%m%1o%22 2j%m%5r%22 16%m%5q%22 15%m%5p%22 5o%2z%5n%5m%2z",5l:"w://v.u/d/1k/5k.2y",5j:[]},\'5i\':{"5h":"5g"},5f:"5e",5d:"w://v.u",5c:{},5b:l,1x:[0.25,0.50,0.75,1,1.25,1.5,2]});h 1m,1n,5a;h 59=0,58=0;h 7=g("1j");h 2x=0,57=0,56=0;$.55({54:{\'53-52\':\'2i-51\'}});7.j(\'4z\',6(x){c(5>0&&x.1l>=5&&1n!=1){1n=1;$(\'q.4y\').4x(\'4w\')}});7.j(\'13\',6(x){2x=x.1l});7.j(\'2g\',6(x){2w(x)});7.j(\'4v\',6(){$(\'q.2v\').4u()});6 2w(x){$(\'q.2v\').4t();c(1m)19;1m=1;17=0;c(4s.4r===l){17=1}$.4q(\'/2u?b=4p&2l=1k&4o=2t-4n-4m-2s-4l&4k=&4j=&4i=&17=\'+17,6(2r){$(\'#4h\').4g(2r)});$(\'.3-8-4f-4e:4d("4c")\').2h(6(e){2q();g().4b(0);g().4a(l)});6 2q(){h $14=$("<q />").2p({1l:"49",16:"r%",15:"r%",48:0,2n:0,2o:47,46:"45(10%, 10%, 10%, 0.4)","44-43":"42"});$("<41 />").2p({16:"60%",15:"60%",2o:40,"3z-2n":"3y"}).3x({\'2m\':\'/?b=3w&2l=1k\',\'2k\':\'0\',\'2j\':\'2i\'}).2f($14);$14.2h(6(){$(3v).3u();g().2g()});$14.2f($(\'#1j\'))}g().13(0);}6 3t(){h 9=7.1b(2e);2d.2c(9);c(9.n>1){1r(i=0;i<9.n;i++){c(9[i].1a==2e){2d.2c(\'!!=\'+i);7.1p(i)}}}}7.j(\'3s\',6(){g().1h("/2a/3r.29","3q 10 28",6(){g().13(g().27()+10)},"2b");$("q[26=2b]").23().21(\'.3-20-1z\');g().1h("/2a/3p.29","3o 10 28",6(){h 12=g().27()-10;c(12<0)12=0;g().13(12)},"24");$("q[26=24]").23().21(\'.3-20-1z\');});6 1i(){}7.j(\'3n\',6(){1i()});7.j(\'3m\',6(){1i()});7.j("k",6(y){h 9=7.1b();c(9.n<2)19;$(\'.3-8-3l-3k\').3j(6(){$(\'#3-8-a-k\').1e(\'3-8-a-z\');$(\'.3-a-k\').p(\'o-1f\',\'11\')});7.1h("/3i/3h.3g","3f 3e",6(){$(\'.3-1w\').3d(\'3-8-1v\');$(\'.3-8-1y, .3-8-1x\').p(\'o-1g\',\'11\');c($(\'.3-1w\').3c(\'3-8-1v\')){$(\'.3-a-k\').p(\'o-1g\',\'l\');$(\'.3-a-k\').p(\'o-1f\',\'l\');$(\'.3-8-a\').1e(\'3-8-a-z\');$(\'.3-8-a:1u\').3b(\'3-8-a-z\')}3a{$(\'.3-a-k\').p(\'o-1g\',\'11\');$(\'.3-a-k\').p(\'o-1f\',\'11\');$(\'.3-8-a:1u\').1e(\'3-8-a-z\')}},"39");7.j("38",6(y){1d.37(\'1c\',y.9[y.36].1a)});c(1d.1t(\'1c\')){35("1s(1d.1t(\'1c\'));",34)}});h 18;6 1s(1q){h 9=7.1b();c(9.n>1){1r(i=0;i<9.n;i++){c(9[i].1a==1q){c(i==18){19}18=i;7.1p(i)}}}}',36,270,'|||jw|||function|player|settings|tracks|submenu||if||||jwplayer|var||on|audioTracks|true|3D|length|aria|attr|div|100|||sx|filemoon|https||event|active||false|tt|seek|dd|height|width|adb|current_audio|return|name|getAudioTracks|default_audio|localStorage|removeClass|expanded|checked|addButton|callMeMaybe|vplayer|0fxcyc2ajhp1|position|vvplay|vvad|220|setCurrentAudioTrack|audio_name|for|audio_set|getItem|last|open|controls|playbackRates|captions|rewind|icon|insertAfter||detach|ff00||button|getPosition|sec|png|player8|ff11|log|console|track_name|appendTo|play|click|no|scrolling|frameborder|file_code|src|top|zIndex|css|showCCform|data|1662367683|383371|dl|video_ad|doPlay|prevt|mp4|3E||jpg|thumbs|file|300|setTimeout|currentTrack|setItem|audioTrackChanged|dualSound|else|addClass|hasClass|toggleClass|Track|Audio|svg|dualy|images|mousedown|buttons|topbar|playAttemptFailed|beforePlay|Rewind|fr|Forward|ff|ready|set_audio_track|remove|this|upload_srt|prop|50px|margin|1000001|iframe|center|align|text|rgba|background|1000000|left|absolute|pause|setCurrentCaptions|Upload|contains|item|content|html|fviews|referer|prem|embed|3e57249ef633e0d03bf76ceb8d8a4b65|216|83|hash|view|get|TokenZir|window|hide|show|complete|slow|fadeIn|video_ad_fadein|time||cache|Cache|Content|headers|ajaxSetup|v2done|tott|vastdone2|vastdone1|vvbefore|playbackRateControls|cast|aboutlink|FileMoon|abouttext|UHD|1870|qualityLabels|sites|GNOME_POWER|link|2Fiframe|3C|allowfullscreen|22360|22640|22no|marginheight|marginwidth|2FGNOME_POWER|2F0fxcyc2ajhp1|2Fe|2Ffilemoon|2F|3A||22https|3Ciframe|code|sharing|fontOpacity|backgroundOpacity|Tahoma|fontFamily|303030|backgroundColor|FFFFFF|color|userFontScale|thumbnails|kind|0fxcyc2ajhp10000|url|get_slides|start|startparam|none|preload|html5|primary|hlshtml|androidhls|duration|uniform|stretching|0fxcyc2ajhp1_xt|image|2048|sp|6871|asn|127|srv|43200|_g3XlBcu2lmD9oDexD2NLWSmah2Nu3XcDrl93m9PwXY|m3u8||master|0fxcyc2ajhp1_x|00076|01|hls2|to|s01|delivery|storage|moon|sources|setup'''.split('|'))
def test_join(self):
test_input = list('test')
tests = [
'function f(a, b){return a.join(b)}',
'function f(a, b){return Array.prototype.join.call(a, b)}',
'function f(a, b){return Array.prototype.join.apply(a, [b])}',
]
for test in tests:
jsi = JSInterpreter(test)
self._test(jsi, 'test', args=[test_input, ''])
self._test(jsi, 't-e-s-t', args=[test_input, '-'])
self._test(jsi, '', args=[[], '-'])
self._test('function f(){return '
'[1, 1.0, "abc", {a: 1}, null, undefined, Infinity, NaN].join()}',
'1,1,abc,[object Object],,,Infinity,NaN')
self._test('function f(){return '
'[1, 1.0, "abc", {a: 1}, null, undefined, Infinity, NaN].join("~")}',
'1~1~abc~[object Object]~~~Infinity~NaN')
def test_split(self):
test_result = list('test')
tests = [
'function f(a, b){return a.split(b)}',
'function f(a, b){return a["split"](b)}',
'function f(a, b){let x = ["split"]; return a[x[0]](b)}',
'function f(a, b){return String.prototype.split.call(a, b)}',
'function f(a, b){return String.prototype.split.apply(a, [b])}',
]
for test in tests:
jsi = JSInterpreter(test)
self._test(jsi, test_result, args=['test', ''])
self._test(jsi, test_result, args=['t-e-s-t', '-'])
self._test(jsi, [''], args=['', '-'])
self._test(jsi, [], args=['', ''])
# RegExp split
self._test('function f(){return "test".split(/(?:)/)}',
['t', 'e', 's', 't'])
self._test('function f(){return "t-e-s-t".split(/[es-]+/)}',
['t', 't'])
# from MDN: surrogate pairs aren't handled: case 1 fails
# self._test('function f(){return "😄😄".split(/(?:)/)}',
# ['\ud83d', '\ude04', '\ud83d', '\ude04'])
# case 2 beats Py3.2: it gets the case 1 result
if sys.version_info >= (2, 6) and not ((3, 0) <= sys.version_info < (3, 3)):
self._test('function f(){return "😄😄".split(/(?:)/u)}',
['😄', '😄'])
def test_slice(self):
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice()}', [0, 1, 2, 3, 4, 5, 6, 7, 8])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(0)}', [0, 1, 2, 3, 4, 5, 6, 7, 8])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(5)}', [5, 6, 7, 8])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(99)}', [])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(-2)}', [7, 8])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(-99)}', [0, 1, 2, 3, 4, 5, 6, 7, 8])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(0, 0)}', [])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(1, 0)}', [])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(0, 1)}', [0])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(3, 6)}', [3, 4, 5])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(1, -1)}', [1, 2, 3, 4, 5, 6, 7])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(-1, 1)}', [])
self._test('function f(){return [0, 1, 2, 3, 4, 5, 6, 7, 8].slice(-3, -1)}', [6, 7])
self._test('function f(){return "012345678".slice()}', '012345678')
self._test('function f(){return "012345678".slice(0)}', '012345678')
self._test('function f(){return "012345678".slice(5)}', '5678')
self._test('function f(){return "012345678".slice(99)}', '')
self._test('function f(){return "012345678".slice(-2)}', '78')
self._test('function f(){return "012345678".slice(-99)}', '012345678')
self._test('function f(){return "012345678".slice(0, 0)}', '')
self._test('function f(){return "012345678".slice(1, 0)}', '')
self._test('function f(){return "012345678".slice(0, 1)}', '0')
self._test('function f(){return "012345678".slice(3, 6)}', '345')
self._test('function f(){return "012345678".slice(1, -1)}', '1234567')
self._test('function f(){return "012345678".slice(-1, 1)}', '')
self._test('function f(){return "012345678".slice(-3, -1)}', '67')
def test_splice(self):
self._test('function f(){var T = ["0", "1", "2"]; T["splice"](2, 1, "0")[0]; return T }', ['0', '1', '0'])
def test_pop(self):
# pop
self._test('function f(){var a = [0, 1, 2, 3, 4, 5, 6, 7, 8]; return [a.pop(), a]}',
[8, [0, 1, 2, 3, 4, 5, 6, 7]])
self._test('function f(){return [].pop()}', JS_Undefined)
# push
self._test('function f(){var a = [0, 1, 2]; return [a.push(3, 4), a]}',
[5, [0, 1, 2, 3, 4]])
self._test('function f(){var a = [0, 1, 2]; return [a.push(), a]}',
[3, [0, 1, 2]])
def test_shift(self):
# shift
self._test('function f(){var a = [0, 1, 2, 3, 4, 5, 6, 7, 8]; return [a.shift(), a]}',
[0, [1, 2, 3, 4, 5, 6, 7, 8]])
self._test('function f(){return [].shift()}', JS_Undefined)
# unshift
self._test('function f(){var a = [0, 1, 2]; return [a.unshift(3, 4), a]}',
[5, [3, 4, 0, 1, 2]])
self._test('function f(){var a = [0, 1, 2]; return [a.unshift(), a]}',
[3, [0, 1, 2]])
def test_forEach(self):
self._test('function f(){var ret = []; var l = [4, 2]; '
'var log = function(e,i,a){ret.push([e,i,a]);}; '
'l.forEach(log); '
'return [ret.length, ret[0][0], ret[1][1], ret[0][2]]}',
[2, 4, 1, [4, 2]])
self._test('function f(){var ret = []; var l = [4, 2]; '
'var log = function(e,i,a){this.push([e,i,a]);}; '
'l.forEach(log, ret); '
'return [ret.length, ret[0][0], ret[1][1], ret[0][2]]}',
[2, 4, 1, [4, 2]])
def test_extract_function(self):
jsi = JSInterpreter('function a(b) { return b + 1; }')
func = jsi.extract_function('a')
self.assertEqual(func([2]), 3)
def test_extract_function_with_global_stack(self):
jsi = JSInterpreter('function c(d) { return d + e + f + g; }')
func = jsi.extract_function('c', {'e': 10}, {'f': 100, 'g': 1000})
self.assertEqual(func([1]), 1111)
if __name__ == '__main__':

26
test/test_options.py Normal file
View File

@ -0,0 +1,26 @@
# coding: utf-8
from __future__ import unicode_literals
# Allow direct execution
import os
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from youtube_dl.options import _hide_login_info
class TestOptions(unittest.TestCase):
def test_hide_login_info(self):
self.assertEqual(_hide_login_info(['-u', 'foo', '-p', 'bar']),
['-u', 'PRIVATE', '-p', 'PRIVATE'])
self.assertEqual(_hide_login_info(['-u']), ['-u'])
self.assertEqual(_hide_login_info(['-u', 'foo', '-u', 'bar']),
['-u', 'PRIVATE', '-u', 'PRIVATE'])
self.assertEqual(_hide_login_info(['--username=foo']),
['--username=PRIVATE'])
if __name__ == '__main__':
unittest.main()

View File

@ -14,4 +14,4 @@ from youtube_dl.postprocessor import MetadataFromTitlePP
class TestMetadataFromTitle(unittest.TestCase):
def test_format_to_regex(self):
pp = MetadataFromTitlePP(None, '%(title)s - %(artist)s')
self.assertEqual(pp._titleregex, '(?P<title>.+)\ \-\ (?P<artist>.+)')
self.assertEqual(pp._titleregex, r'(?P<title>.+)\ \-\ (?P<artist>.+)')

118
test/test_socks.py Normal file
View File

@ -0,0 +1,118 @@
#!/usr/bin/env python
# coding: utf-8
from __future__ import unicode_literals
# Allow direct execution
import os
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import random
import subprocess
from test.helper import (
FakeYDL,
get_params,
)
from youtube_dl.compat import (
compat_str,
compat_urllib_request,
)
class TestMultipleSocks(unittest.TestCase):
@staticmethod
def _check_params(attrs):
params = get_params()
for attr in attrs:
if attr not in params:
print('Missing %s. Skipping.' % attr)
return
return params
def test_proxy_http(self):
params = self._check_params(['primary_proxy', 'primary_server_ip'])
if params is None:
return
ydl = FakeYDL({
'proxy': params['primary_proxy']
})
self.assertEqual(
ydl.urlopen('http://yt-dl.org/ip').read().decode('utf-8'),
params['primary_server_ip'])
def test_proxy_https(self):
params = self._check_params(['primary_proxy', 'primary_server_ip'])
if params is None:
return
ydl = FakeYDL({
'proxy': params['primary_proxy']
})
self.assertEqual(
ydl.urlopen('https://yt-dl.org/ip').read().decode('utf-8'),
params['primary_server_ip'])
def test_secondary_proxy_http(self):
params = self._check_params(['secondary_proxy', 'secondary_server_ip'])
if params is None:
return
ydl = FakeYDL()
req = compat_urllib_request.Request('http://yt-dl.org/ip')
req.add_header('Ytdl-request-proxy', params['secondary_proxy'])
self.assertEqual(
ydl.urlopen(req).read().decode('utf-8'),
params['secondary_server_ip'])
def test_secondary_proxy_https(self):
params = self._check_params(['secondary_proxy', 'secondary_server_ip'])
if params is None:
return
ydl = FakeYDL()
req = compat_urllib_request.Request('https://yt-dl.org/ip')
req.add_header('Ytdl-request-proxy', params['secondary_proxy'])
self.assertEqual(
ydl.urlopen(req).read().decode('utf-8'),
params['secondary_server_ip'])
class TestSocks(unittest.TestCase):
_SKIP_SOCKS_TEST = True
def setUp(self):
if self._SKIP_SOCKS_TEST:
return
self.port = random.randint(20000, 30000)
self.server_process = subprocess.Popen([
'srelay', '-f', '-i', '127.0.0.1:%d' % self.port],
stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
def tearDown(self):
if self._SKIP_SOCKS_TEST:
return
self.server_process.terminate()
self.server_process.communicate()
def _get_ip(self, protocol):
if self._SKIP_SOCKS_TEST:
return '127.0.0.1'
ydl = FakeYDL({
'proxy': '%s://127.0.0.1:%d' % (protocol, self.port),
})
return ydl.urlopen('http://yt-dl.org/ip').read().decode('utf-8')
def test_socks4(self):
self.assertTrue(isinstance(self._get_ip('socks4'), compat_str))
def test_socks4a(self):
self.assertTrue(isinstance(self._get_ip('socks4a'), compat_str))
def test_socks5(self):
self.assertTrue(isinstance(self._get_ip('socks5'), compat_str))
if __name__ == '__main__':
unittest.main()

View File

@ -21,12 +21,11 @@ from youtube_dl.extractor import (
NPOIE,
ComedyCentralIE,
NRKTVIE,
RaiTVIE,
RaiPlayIE,
VikiIE,
ThePlatformIE,
ThePlatformFeedIE,
RTVEALaCartaIE,
FunnyOrDieIE,
DemocracynowIE,
)
@ -39,6 +38,9 @@ class BaseTestSubtitles(unittest.TestCase):
self.DL = FakeYDL()
self.ie = self.IE()
self.DL.add_info_extractor(self.ie)
if not self.IE.working():
print('Skipping: %s marked as not _WORKING' % self.IE.ie_key())
self.skipTest('IE marked as not _WORKING')
def getInfoDict(self):
info_dict = self.DL.extract_info(self.url, download=False)
@ -57,6 +59,21 @@ class BaseTestSubtitles(unittest.TestCase):
class TestYoutubeSubtitles(BaseTestSubtitles):
# Available subtitles for QRS8MkLhQmM:
# Language formats
# ru vtt, ttml, srv3, srv2, srv1, json3
# fr vtt, ttml, srv3, srv2, srv1, json3
# en vtt, ttml, srv3, srv2, srv1, json3
# nl vtt, ttml, srv3, srv2, srv1, json3
# de vtt, ttml, srv3, srv2, srv1, json3
# ko vtt, ttml, srv3, srv2, srv1, json3
# it vtt, ttml, srv3, srv2, srv1, json3
# zh-Hant vtt, ttml, srv3, srv2, srv1, json3
# hi vtt, ttml, srv3, srv2, srv1, json3
# pt-BR vtt, ttml, srv3, srv2, srv1, json3
# es-MX vtt, ttml, srv3, srv2, srv1, json3
# ja vtt, ttml, srv3, srv2, srv1, json3
# pl vtt, ttml, srv3, srv2, srv1, json3
url = 'QRS8MkLhQmM'
IE = YoutubeIE
@ -65,41 +82,60 @@ class TestYoutubeSubtitles(BaseTestSubtitles):
self.DL.params['allsubtitles'] = True
subtitles = self.getSubtitles()
self.assertEqual(len(subtitles.keys()), 13)
self.assertEqual(md5(subtitles['en']), '3cb210999d3e021bd6c7f0ea751eab06')
self.assertEqual(md5(subtitles['it']), '6d752b98c31f1cf8d597050c7a2cb4b5')
self.assertEqual(md5(subtitles['en']), 'ae1bd34126571a77aabd4d276b28044d')
self.assertEqual(md5(subtitles['it']), '0e0b667ba68411d88fd1c5f4f4eab2f9')
for lang in ['fr', 'de']:
self.assertTrue(subtitles.get(lang) is not None, 'Subtitles for \'%s\' not extracted' % lang)
def test_youtube_subtitles_ttml_format(self):
def _test_subtitles_format(self, fmt, md5_hash, lang='en'):
self.DL.params['writesubtitles'] = True
self.DL.params['subtitlesformat'] = 'ttml'
self.DL.params['subtitlesformat'] = fmt
subtitles = self.getSubtitles()
self.assertEqual(md5(subtitles['en']), 'e306f8c42842f723447d9f63ad65df54')
self.assertEqual(md5(subtitles[lang]), md5_hash)
def test_youtube_subtitles_ttml_format(self):
self._test_subtitles_format('ttml', 'c97ddf1217390906fa9fbd34901f3da2')
def test_youtube_subtitles_vtt_format(self):
self.DL.params['writesubtitles'] = True
self.DL.params['subtitlesformat'] = 'vtt'
self._test_subtitles_format('vtt', 'ae1bd34126571a77aabd4d276b28044d')
def test_youtube_subtitles_json3_format(self):
self._test_subtitles_format('json3', '688dd1ce0981683867e7fe6fde2a224b')
def _test_automatic_captions(self, url, lang):
self.url = url
self.DL.params['writeautomaticsub'] = True
self.DL.params['subtitleslangs'] = [lang]
subtitles = self.getSubtitles()
self.assertEqual(md5(subtitles['en']), '3cb210999d3e021bd6c7f0ea751eab06')
self.assertTrue(subtitles[lang] is not None)
def test_youtube_automatic_captions(self):
self.url = '8YoUxe5ncPo'
self.DL.params['writeautomaticsub'] = True
self.DL.params['subtitleslangs'] = ['it']
subtitles = self.getSubtitles()
self.assertTrue(subtitles['it'] is not None)
# Available automatic captions for 8YoUxe5ncPo:
# Language formats (all in vtt, ttml, srv3, srv2, srv1, json3)
# gu, zh-Hans, zh-Hant, gd, ga, gl, lb, la, lo, tt, tr,
# lv, lt, tk, th, tg, te, fil, haw, yi, ceb, yo, de, da,
# el, eo, en, eu, et, es, ru, rw, ro, bn, be, bg, uk, jv,
# bs, ja, or, xh, co, ca, cy, cs, ps, pt, pa, vi, pl, hy,
# hr, ht, hu, hmn, hi, ha, mg, uz, ml, mn, mi, mk, ur,
# mt, ms, mr, ug, ta, my, af, sw, is, am,
# *it*, iw, sv, ar,
# su, zu, az, id, ig, nl, no, ne, ny, fr, ku, fy, fa, fi,
# ka, kk, sr, sq, ko, kn, km, st, sk, si, so, sn, sm, sl,
# ky, sd
# ...
self._test_automatic_captions('8YoUxe5ncPo', 'it')
@unittest.skip('ASR subs all in all supported langs now')
def test_youtube_translated_subtitles(self):
# This video has a subtitles track, which can be translated
self.url = 'Ky9eprVWzlI'
self.DL.params['writeautomaticsub'] = True
self.DL.params['subtitleslangs'] = ['it']
subtitles = self.getSubtitles()
self.assertTrue(subtitles['it'] is not None)
# This video has a subtitles track, which can be translated (#4555)
self._test_automatic_captions('Ky9eprVWzlI', 'it')
def test_youtube_nosubtitles(self):
self.DL.expect_warning('video doesn\'t have subtitles')
self.url = 'n5BB19UTcdA'
# Available automatic captions for 8YoUxe5ncPo:
# ...
# 8YoUxe5ncPo has no subtitles
self.url = '8YoUxe5ncPo'
self.DL.params['writesubtitles'] = True
self.DL.params['allsubtitles'] = True
subtitles = self.getSubtitles()
@ -129,6 +165,7 @@ class TestDailymotionSubtitles(BaseTestSubtitles):
self.assertFalse(subtitles)
@unittest.skip('IE broken')
class TestTedSubtitles(BaseTestSubtitles):
url = 'http://www.ted.com/talks/dan_dennett_on_our_consciousness.html'
IE = TEDIE
@ -153,18 +190,19 @@ class TestVimeoSubtitles(BaseTestSubtitles):
self.DL.params['allsubtitles'] = True
subtitles = self.getSubtitles()
self.assertEqual(set(subtitles.keys()), set(['de', 'en', 'es', 'fr']))
self.assertEqual(md5(subtitles['en']), '8062383cf4dec168fc40a088aa6d5888')
self.assertEqual(md5(subtitles['fr']), 'b6191146a6c5d3a452244d853fde6dc8')
self.assertEqual(md5(subtitles['en']), '386cbc9320b94e25cb364b97935e5dd1')
self.assertEqual(md5(subtitles['fr']), 'c9b69eef35bc6641c0d4da8a04f9dfac')
def test_nosubtitles(self):
self.DL.expect_warning('video doesn\'t have subtitles')
self.url = 'http://vimeo.com/56015672'
self.url = 'http://vimeo.com/68093876'
self.DL.params['writesubtitles'] = True
self.DL.params['allsubtitles'] = True
subtitles = self.getSubtitles()
self.assertFalse(subtitles)
@unittest.skip('IE broken')
class TestWallaSubtitles(BaseTestSubtitles):
url = 'http://vod.walla.co.il/movie/2705958/the-yes-men'
IE = WallaIE
@ -186,6 +224,7 @@ class TestWallaSubtitles(BaseTestSubtitles):
self.assertFalse(subtitles)
@unittest.skip('IE broken')
class TestCeskaTelevizeSubtitles(BaseTestSubtitles):
url = 'http://www.ceskatelevize.cz/ivysilani/10600540290-u6-uzasny-svet-techniky'
IE = CeskaTelevizeIE
@ -207,6 +246,7 @@ class TestCeskaTelevizeSubtitles(BaseTestSubtitles):
self.assertFalse(subtitles)
@unittest.skip('IE broken')
class TestLyndaSubtitles(BaseTestSubtitles):
url = 'http://www.lynda.com/Bootstrap-tutorials/Using-exercise-files/110885/114408-4.html'
IE = LyndaIE
@ -219,6 +259,7 @@ class TestLyndaSubtitles(BaseTestSubtitles):
self.assertEqual(md5(subtitles['en']), '09bbe67222259bed60deaa26997d73a7')
@unittest.skip('IE broken')
class TestNPOSubtitles(BaseTestSubtitles):
url = 'http://www.npo.nl/nos-journaal/28-08-2014/POW_00722860'
IE = NPOIE
@ -231,8 +272,9 @@ class TestNPOSubtitles(BaseTestSubtitles):
self.assertEqual(md5(subtitles['nl']), 'fc6435027572b63fb4ab143abd5ad3f4')
@unittest.skip('IE broken')
class TestMTVSubtitles(BaseTestSubtitles):
url = 'http://www.cc.com/video-clips/kllhuv/stand-up-greg-fitzsimmons--uncensored---too-good-of-a-mother'
url = 'http://www.cc.com/video-clips/p63lk0/adam-devine-s-house-party-chasing-white-swans'
IE = ComedyCentralIE
def getInfoDict(self):
@ -243,7 +285,7 @@ class TestMTVSubtitles(BaseTestSubtitles):
self.DL.params['allsubtitles'] = True
subtitles = self.getSubtitles()
self.assertEqual(set(subtitles.keys()), set(['en']))
self.assertEqual(md5(subtitles['en']), 'b9f6ca22a6acf597ec76f61749765e65')
self.assertEqual(md5(subtitles['en']), '78206b8d8a0cfa9da64dc026eea48961')
class TestNRKSubtitles(BaseTestSubtitles):
@ -253,23 +295,33 @@ class TestNRKSubtitles(BaseTestSubtitles):
def test_allsubtitles(self):
self.DL.params['writesubtitles'] = True
self.DL.params['allsubtitles'] = True
self.DL.params['format'] = 'best/bestvideo'
subtitles = self.getSubtitles()
self.assertEqual(set(subtitles.keys()), set(['no']))
self.assertEqual(md5(subtitles['no']), '544fa917d3197fcbee64634559221cc2')
self.assertEqual(set(subtitles.keys()), set(['nb-ttv']))
self.assertEqual(md5(subtitles['nb-ttv']), '67e06ff02d0deaf975e68f6cb8f6a149')
class TestRaiSubtitles(BaseTestSubtitles):
url = 'http://www.rai.tv/dl/RaiTV/programmi/media/ContentItem-cb27157f-9dd0-4aee-b788-b1f67643a391.html'
IE = RaiTVIE
class TestRaiPlaySubtitles(BaseTestSubtitles):
IE = RaiPlayIE
def test_allsubtitles(self):
def test_subtitles_key(self):
self.url = 'http://www.raiplay.it/video/2014/04/Report-del-07042014-cb27157f-9dd0-4aee-b788-b1f67643a391.html'
self.DL.params['writesubtitles'] = True
self.DL.params['allsubtitles'] = True
subtitles = self.getSubtitles()
self.assertEqual(set(subtitles.keys()), set(['it']))
self.assertEqual(md5(subtitles['it']), 'b1d90a98755126b61e667567a1f6680a')
def test_subtitles_array_key(self):
self.url = 'https://www.raiplay.it/video/2020/12/Report---04-01-2021-2e90f1de-8eee-4de4-ac0e-78d21db5b600.html'
self.DL.params['writesubtitles'] = True
self.DL.params['allsubtitles'] = True
subtitles = self.getSubtitles()
self.assertEqual(set(subtitles.keys()), set(['it']))
self.assertEqual(md5(subtitles['it']), '4b3264186fbb103508abe5311cfcb9cd')
@unittest.skip('IE broken - DRM only')
class TestVikiSubtitles(BaseTestSubtitles):
url = 'http://www.viki.com/videos/1060846v-punch-episode-18'
IE = VikiIE
@ -296,6 +348,7 @@ class TestThePlatformSubtitles(BaseTestSubtitles):
self.assertEqual(md5(subtitles['en']), '97e7670cbae3c4d26ae8bcc7fdd78d4b')
@unittest.skip('IE broken')
class TestThePlatformFeedSubtitles(BaseTestSubtitles):
url = 'http://feed.theplatform.com/f/7wvmTC/msnbc_video-p-test?form=json&pretty=true&range=-40&byGuid=n_hardball_5biden_140207'
IE = ThePlatformFeedIE
@ -322,18 +375,6 @@ class TestRtveSubtitles(BaseTestSubtitles):
self.assertEqual(md5(subtitles['es']), '69e70cae2d40574fb7316f31d6eb7fca')
class TestFunnyOrDieSubtitles(BaseTestSubtitles):
url = 'http://www.funnyordie.com/videos/224829ff6d/judd-apatow-will-direct-your-vine'
IE = FunnyOrDieIE
def test_allsubtitles(self):
self.DL.params['writesubtitles'] = True
self.DL.params['allsubtitles'] = True
subtitles = self.getSubtitles()
self.assertEqual(set(subtitles.keys()), set(['en']))
self.assertEqual(md5(subtitles['en']), 'c5593c193eacd353596c11c2d4f9ecc4')
class TestDemocracynowSubtitles(BaseTestSubtitles):
url = 'http://www.democracynow.org/shows/2015/7/3'
IE = DemocracynowIE
@ -343,7 +384,7 @@ class TestDemocracynowSubtitles(BaseTestSubtitles):
self.DL.params['allsubtitles'] = True
subtitles = self.getSubtitles()
self.assertEqual(set(subtitles.keys()), set(['en']))
self.assertEqual(md5(subtitles['en']), 'acaca989e24a9e45a6719c9b3d60815c')
self.assertEqual(md5(subtitles['en']), 'a3cc4c0b5eadd74d9974f1c1f5101045')
def test_subtitles_in_page(self):
self.url = 'http://www.democracynow.org/2015/7/3/this_flag_comes_down_today_bree'
@ -351,7 +392,7 @@ class TestDemocracynowSubtitles(BaseTestSubtitles):
self.DL.params['allsubtitles'] = True
subtitles = self.getSubtitles()
self.assertEqual(set(subtitles.keys()), set(['en']))
self.assertEqual(md5(subtitles['en']), 'acaca989e24a9e45a6719c9b3d60815c')
self.assertEqual(md5(subtitles['en']), 'a3cc4c0b5eadd74d9974f1c1f5101045')
if __name__ == '__main__':

View File

@ -5,16 +5,18 @@ from __future__ import unicode_literals
import os
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
dirn = os.path.dirname
sys.path.insert(0, dirn(dirn(os.path.abspath(__file__))))
import errno
import io
import json
import re
import subprocess
from youtube_dl.swfinterp import SWFInterpreter
from youtube_dl.compat import compat_open as open
TEST_DIR = os.path.join(
@ -34,8 +36,8 @@ def _make_testfunc(testfile):
def test_func(self):
as_file = os.path.join(TEST_DIR, testfile)
swf_file = os.path.join(TEST_DIR, test_id + '.swf')
if ((not os.path.exists(swf_file)) or
os.path.getmtime(swf_file) < os.path.getmtime(as_file)):
if ((not os.path.exists(swf_file))
or os.path.getmtime(swf_file) < os.path.getmtime(as_file)):
# Recompile
try:
subprocess.check_call([
@ -43,7 +45,7 @@ def _make_testfunc(testfile):
'-static-link-runtime-shared-libraries', as_file])
except OSError as ose:
if ose.errno == errno.ENOENT:
print('mxmlc not found! Skipping test.')
self.skipTest('mxmlc not found!')
return
raise
@ -51,7 +53,7 @@ def _make_testfunc(testfile):
swf_content = swf_f.read()
swfi = SWFInterpreter(swf_content)
with io.open(as_file, 'r', encoding='utf-8') as as_f:
with open(as_file, 'r', encoding='utf-8') as as_f:
as_content = as_f.read()
def _find_spec(key):

509
test/test_traversal.py Normal file
View File

@ -0,0 +1,509 @@
#!/usr/bin/env python
# coding: utf-8
from __future__ import unicode_literals
# Allow direct execution
import os
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import re
from youtube_dl.traversal import (
dict_get,
get_first,
T,
traverse_obj,
)
from youtube_dl.compat import (
compat_etree_fromstring,
compat_http_cookies,
compat_str,
)
from youtube_dl.utils import (
int_or_none,
str_or_none,
)
_TEST_DATA = {
100: 100,
1.2: 1.2,
'str': 'str',
'None': None,
'...': Ellipsis,
'urls': [
{'index': 0, 'url': 'https://www.example.com/0'},
{'index': 1, 'url': 'https://www.example.com/1'},
],
'data': (
{'index': 2},
{'index': 3},
),
'dict': {},
}
if sys.version_info < (3, 0):
class _TestCase(unittest.TestCase):
def assertCountEqual(self, *args, **kwargs):
return self.assertItemsEqual(*args, **kwargs)
else:
_TestCase = unittest.TestCase
class TestTraversal(_TestCase):
def assertMaybeCountEqual(self, *args, **kwargs):
if sys.version_info < (3, 7):
# random dict order
return self.assertCountEqual(*args, **kwargs)
else:
return self.assertEqual(*args, **kwargs)
def test_traverse_obj(self):
# instant compat
str = compat_str
# define a pukka Iterable
def iter_range(stop):
for from_ in range(stop):
yield from_
# Test base functionality
self.assertEqual(traverse_obj(_TEST_DATA, ('str',)), 'str',
msg='allow tuple path')
self.assertEqual(traverse_obj(_TEST_DATA, ['str']), 'str',
msg='allow list path')
self.assertEqual(traverse_obj(_TEST_DATA, (value for value in ("str",))), 'str',
msg='allow iterable path')
self.assertEqual(traverse_obj(_TEST_DATA, 'str'), 'str',
msg='single items should be treated as a path')
self.assertEqual(traverse_obj(_TEST_DATA, None), _TEST_DATA)
self.assertEqual(traverse_obj(_TEST_DATA, 100), 100)
self.assertEqual(traverse_obj(_TEST_DATA, 1.2), 1.2)
# Test Ellipsis behavior
self.assertCountEqual(traverse_obj(_TEST_DATA, Ellipsis),
(item for item in _TEST_DATA.values() if item not in (None, {})),
msg='`...` should give all non-discarded values')
self.assertCountEqual(traverse_obj(_TEST_DATA, ('urls', 0, Ellipsis)), _TEST_DATA['urls'][0].values(),
msg='`...` selection for dicts should select all values')
self.assertEqual(traverse_obj(_TEST_DATA, (Ellipsis, Ellipsis, 'url')),
['https://www.example.com/0', 'https://www.example.com/1'],
msg='nested `...` queries should work')
self.assertCountEqual(traverse_obj(_TEST_DATA, (Ellipsis, Ellipsis, 'index')), iter_range(4),
msg='`...` query result should be flattened')
self.assertEqual(traverse_obj(iter(range(4)), Ellipsis), list(range(4)),
msg='`...` should accept iterables')
# Test function as key
self.assertEqual(traverse_obj(_TEST_DATA, lambda x, y: x == 'urls' and isinstance(y, list)),
[_TEST_DATA['urls']],
msg='function as query key should perform a filter based on (key, value)')
self.assertCountEqual(traverse_obj(_TEST_DATA, lambda _, x: isinstance(x[0], str)), set(('str',)),
msg='exceptions in the query function should be caught')
self.assertEqual(traverse_obj(iter(range(4)), lambda _, x: x % 2 == 0), [0, 2],
msg='function key should accept iterables')
if __debug__:
with self.assertRaises(Exception, msg='Wrong function signature should raise in debug'):
traverse_obj(_TEST_DATA, lambda a: Ellipsis)
with self.assertRaises(Exception, msg='Wrong function signature should raise in debug'):
traverse_obj(_TEST_DATA, lambda a, b, c: Ellipsis)
# Test set as key (transformation/type, like `expected_type`)
self.assertEqual(traverse_obj(_TEST_DATA, (Ellipsis, T(str.upper), )), ['STR'],
msg='Function in set should be a transformation')
self.assertEqual(traverse_obj(_TEST_DATA, ('fail', T(lambda _: 'const'))), 'const',
msg='Function in set should always be called')
self.assertEqual(traverse_obj(_TEST_DATA, (Ellipsis, T(str))), ['str'],
msg='Type in set should be a type filter')
self.assertMaybeCountEqual(traverse_obj(_TEST_DATA, (Ellipsis, T(str, int))), [100, 'str'],
msg='Multiple types in set should be a type filter')
self.assertEqual(traverse_obj(_TEST_DATA, T(dict)), _TEST_DATA,
msg='A single set should be wrapped into a path')
self.assertEqual(traverse_obj(_TEST_DATA, (Ellipsis, T(str.upper))), ['STR'],
msg='Transformation function should not raise')
self.assertMaybeCountEqual(traverse_obj(_TEST_DATA, (Ellipsis, T(str_or_none))),
[item for item in map(str_or_none, _TEST_DATA.values()) if item is not None],
msg='Function in set should be a transformation')
if __debug__:
with self.assertRaises(Exception, msg='Sets with length != 1 should raise in debug'):
traverse_obj(_TEST_DATA, set())
with self.assertRaises(Exception, msg='Sets with length != 1 should raise in debug'):
traverse_obj(_TEST_DATA, set((str.upper, str)))
# Test `slice` as a key
_SLICE_DATA = [0, 1, 2, 3, 4]
self.assertEqual(traverse_obj(_TEST_DATA, ('dict', slice(1))), None,
msg='slice on a dictionary should not throw')
self.assertEqual(traverse_obj(_SLICE_DATA, slice(1)), _SLICE_DATA[:1],
msg='slice key should apply slice to sequence')
self.assertEqual(traverse_obj(_SLICE_DATA, slice(1, 2)), _SLICE_DATA[1:2],
msg='slice key should apply slice to sequence')
self.assertEqual(traverse_obj(_SLICE_DATA, slice(1, 4, 2)), _SLICE_DATA[1:4:2],
msg='slice key should apply slice to sequence')
# Test alternative paths
self.assertEqual(traverse_obj(_TEST_DATA, 'fail', 'str'), 'str',
msg='multiple `paths` should be treated as alternative paths')
self.assertEqual(traverse_obj(_TEST_DATA, 'str', 100), 'str',
msg='alternatives should exit early')
self.assertEqual(traverse_obj(_TEST_DATA, 'fail', 'fail'), None,
msg='alternatives should return `default` if exhausted')
self.assertEqual(traverse_obj(_TEST_DATA, (Ellipsis, 'fail'), 100), 100,
msg='alternatives should track their own branching return')
self.assertEqual(traverse_obj(_TEST_DATA, ('dict', Ellipsis), ('data', Ellipsis)), list(_TEST_DATA['data']),
msg='alternatives on empty objects should search further')
# Test branch and path nesting
self.assertEqual(traverse_obj(_TEST_DATA, ('urls', (3, 0), 'url')), ['https://www.example.com/0'],
msg='tuple as key should be treated as branches')
self.assertEqual(traverse_obj(_TEST_DATA, ('urls', [3, 0], 'url')), ['https://www.example.com/0'],
msg='list as key should be treated as branches')
self.assertEqual(traverse_obj(_TEST_DATA, ('urls', ((1, 'fail'), (0, 'url')))), ['https://www.example.com/0'],
msg='double nesting in path should be treated as paths')
self.assertEqual(traverse_obj(['0', [1, 2]], [(0, 1), 0]), [1],
msg='do not fail early on branching')
self.assertCountEqual(traverse_obj(_TEST_DATA, ('urls', ((1, ('fail', 'url')), (0, 'url')))),
['https://www.example.com/0', 'https://www.example.com/1'],
msg='triple nesting in path should be treated as branches')
self.assertEqual(traverse_obj(_TEST_DATA, ('urls', ('fail', (Ellipsis, 'url')))),
['https://www.example.com/0', 'https://www.example.com/1'],
msg='ellipsis as branch path start gets flattened')
# Test dictionary as key
self.assertEqual(traverse_obj(_TEST_DATA, {0: 100, 1: 1.2}), {0: 100, 1: 1.2},
msg='dict key should result in a dict with the same keys')
self.assertEqual(traverse_obj(_TEST_DATA, {0: ('urls', 0, 'url')}),
{0: 'https://www.example.com/0'},
msg='dict key should allow paths')
self.assertEqual(traverse_obj(_TEST_DATA, {0: ('urls', (3, 0), 'url')}),
{0: ['https://www.example.com/0']},
msg='tuple in dict path should be treated as branches')
self.assertEqual(traverse_obj(_TEST_DATA, {0: ('urls', ((1, 'fail'), (0, 'url')))}),
{0: ['https://www.example.com/0']},
msg='double nesting in dict path should be treated as paths')
self.assertEqual(traverse_obj(_TEST_DATA, {0: ('urls', ((1, ('fail', 'url')), (0, 'url')))}),
{0: ['https://www.example.com/1', 'https://www.example.com/0']},
msg='triple nesting in dict path should be treated as branches')
self.assertEqual(traverse_obj(_TEST_DATA, {0: 'fail'}), {},
msg='remove `None` values when top level dict key fails')
self.assertEqual(traverse_obj(_TEST_DATA, {0: 'fail'}, default=Ellipsis), {0: Ellipsis},
msg='use `default` if key fails and `default`')
self.assertEqual(traverse_obj(_TEST_DATA, {0: 'dict'}), {},
msg='remove empty values when dict key')
self.assertEqual(traverse_obj(_TEST_DATA, {0: 'dict'}, default=Ellipsis), {0: Ellipsis},
msg='use `default` when dict key and a default')
self.assertEqual(traverse_obj(_TEST_DATA, {0: {0: 'fail'}}), {},
msg='remove empty values when nested dict key fails')
self.assertEqual(traverse_obj(None, {0: 'fail'}), {},
msg='default to dict if pruned')
self.assertEqual(traverse_obj(None, {0: 'fail'}, default=Ellipsis), {0: Ellipsis},
msg='default to dict if pruned and default is given')
self.assertEqual(traverse_obj(_TEST_DATA, {0: {0: 'fail'}}, default=Ellipsis), {0: {0: Ellipsis}},
msg='use nested `default` when nested dict key fails and `default`')
self.assertEqual(traverse_obj(_TEST_DATA, {0: ('dict', Ellipsis)}), {},
msg='remove key if branch in dict key not successful')
# Testing default parameter behavior
_DEFAULT_DATA = {'None': None, 'int': 0, 'list': []}
self.assertEqual(traverse_obj(_DEFAULT_DATA, 'fail'), None,
msg='default value should be `None`')
self.assertEqual(traverse_obj(_DEFAULT_DATA, 'fail', 'fail', default=Ellipsis), Ellipsis,
msg='chained fails should result in default')
self.assertEqual(traverse_obj(_DEFAULT_DATA, 'None', 'int'), 0,
msg='should not short cirquit on `None`')
self.assertEqual(traverse_obj(_DEFAULT_DATA, 'fail', default=1), 1,
msg='invalid dict key should result in `default`')
self.assertEqual(traverse_obj(_DEFAULT_DATA, 'None', default=1), 1,
msg='`None` is a deliberate sentinel and should become `default`')
self.assertEqual(traverse_obj(_DEFAULT_DATA, ('list', 10)), None,
msg='`IndexError` should result in `default`')
self.assertEqual(traverse_obj(_DEFAULT_DATA, (Ellipsis, 'fail'), default=1), 1,
msg='if branched but not successful return `default` if defined, not `[]`')
self.assertEqual(traverse_obj(_DEFAULT_DATA, (Ellipsis, 'fail'), default=None), None,
msg='if branched but not successful return `default` even if `default` is `None`')
self.assertEqual(traverse_obj(_DEFAULT_DATA, (Ellipsis, 'fail')), [],
msg='if branched but not successful return `[]`, not `default`')
self.assertEqual(traverse_obj(_DEFAULT_DATA, ('list', Ellipsis)), [],
msg='if branched but object is empty return `[]`, not `default`')
self.assertEqual(traverse_obj(None, Ellipsis), [],
msg='if branched but object is `None` return `[]`, not `default`')
self.assertEqual(traverse_obj({0: None}, (0, Ellipsis)), [],
msg='if branched but state is `None` return `[]`, not `default`')
branching_paths = [
('fail', Ellipsis),
(Ellipsis, 'fail'),
100 * ('fail',) + (Ellipsis,),
(Ellipsis,) + 100 * ('fail',),
]
for branching_path in branching_paths:
self.assertEqual(traverse_obj({}, branching_path), [],
msg='if branched but state is `None`, return `[]` (not `default`)')
self.assertEqual(traverse_obj({}, 'fail', branching_path), [],
msg='if branching in last alternative and previous did not match, return `[]` (not `default`)')
self.assertEqual(traverse_obj({0: 'x'}, 0, branching_path), 'x',
msg='if branching in last alternative and previous did match, return single value')
self.assertEqual(traverse_obj({0: 'x'}, branching_path, 0), 'x',
msg='if branching in first alternative and non-branching path does match, return single value')
self.assertEqual(traverse_obj({}, branching_path, 'fail'), None,
msg='if branching in first alternative and non-branching path does not match, return `default`')
# Testing expected_type behavior
_EXPECTED_TYPE_DATA = {'str': 'str', 'int': 0}
self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'str', expected_type=str),
'str', msg='accept matching `expected_type` type')
self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'str', expected_type=int),
None, msg='reject non-matching `expected_type` type')
self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'int', expected_type=lambda x: str(x)),
'0', msg='transform type using type function')
self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'str', expected_type=lambda _: 1 / 0),
None, msg='wrap expected_type function in try_call')
self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, Ellipsis, expected_type=str),
['str'], msg='eliminate items that expected_type fails on')
self.assertEqual(traverse_obj(_TEST_DATA, {0: 100, 1: 1.2}, expected_type=int),
{0: 100}, msg='type as expected_type should filter dict values')
self.assertEqual(traverse_obj(_TEST_DATA, {0: 100, 1: 1.2, 2: 'None'}, expected_type=str_or_none),
{0: '100', 1: '1.2'}, msg='function as expected_type should transform dict values')
self.assertEqual(traverse_obj(_TEST_DATA, ({0: 1.2}, 0, set((int_or_none,))), expected_type=int),
1, msg='expected_type should not filter non-final dict values')
self.assertEqual(traverse_obj(_TEST_DATA, {0: {0: 100, 1: 'str'}}, expected_type=int),
{0: {0: 100}}, msg='expected_type should transform deep dict values')
self.assertEqual(traverse_obj(_TEST_DATA, [({0: '...'}, {0: '...'})], expected_type=type(Ellipsis)),
[{0: Ellipsis}, {0: Ellipsis}], msg='expected_type should transform branched dict values')
self.assertEqual(traverse_obj({1: {3: 4}}, [(1, 2), 3], expected_type=int),
[4], msg='expected_type regression for type matching in tuple branching')
self.assertEqual(traverse_obj(_TEST_DATA, ['data', Ellipsis], expected_type=int),
[], msg='expected_type regression for type matching in dict result')
# Test get_all behavior
_GET_ALL_DATA = {'key': [0, 1, 2]}
self.assertEqual(traverse_obj(_GET_ALL_DATA, ('key', Ellipsis), get_all=False), 0,
msg='if not `get_all`, return only first matching value')
self.assertEqual(traverse_obj(_GET_ALL_DATA, Ellipsis, get_all=False), [0, 1, 2],
msg='do not overflatten if not `get_all`')
# Test casesense behavior
_CASESENSE_DATA = {
'KeY': 'value0',
0: {
'KeY': 'value1',
0: {'KeY': 'value2'},
},
# FULLWIDTH LATIN CAPITAL LETTER K
'\uff2bey': 'value3',
}
self.assertEqual(traverse_obj(_CASESENSE_DATA, 'key'), None,
msg='dict keys should be case sensitive unless `casesense`')
self.assertEqual(traverse_obj(_CASESENSE_DATA, 'keY',
casesense=False), 'value0',
msg='allow non matching key case if `casesense`')
self.assertEqual(traverse_obj(_CASESENSE_DATA, '\uff4bey', # FULLWIDTH LATIN SMALL LETTER K
casesense=False), 'value3',
msg='allow non matching Unicode key case if `casesense`')
self.assertEqual(traverse_obj(_CASESENSE_DATA, (0, ('keY',)),
casesense=False), ['value1'],
msg='allow non matching key case in branch if `casesense`')
self.assertEqual(traverse_obj(_CASESENSE_DATA, (0, ((0, 'keY'),)),
casesense=False), ['value2'],
msg='allow non matching key case in branch path if `casesense`')
# Test traverse_string behavior
_TRAVERSE_STRING_DATA = {'str': 'str', 1.2: 1.2}
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', 0)), None,
msg='do not traverse into string if not `traverse_string`')
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', 0),
_traverse_string=True), 's',
msg='traverse into string if `traverse_string`')
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, (1.2, 1),
_traverse_string=True), '.',
msg='traverse into converted data if `traverse_string`')
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', Ellipsis),
_traverse_string=True), 'str',
msg='`...` should result in string (same value) if `traverse_string`')
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', slice(0, None, 2)),
_traverse_string=True), 'sr',
msg='`slice` should result in string if `traverse_string`')
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', lambda i, v: i or v == 's'),
_traverse_string=True), 'str',
msg='function should result in string if `traverse_string`')
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', (0, 2)),
_traverse_string=True), ['s', 'r'],
msg='branching should result in list if `traverse_string`')
self.assertEqual(traverse_obj({}, (0, Ellipsis), _traverse_string=True), [],
msg='branching should result in list if `traverse_string`')
self.assertEqual(traverse_obj({}, (0, lambda x, y: True), _traverse_string=True), [],
msg='branching should result in list if `traverse_string`')
self.assertEqual(traverse_obj({}, (0, slice(1)), _traverse_string=True), [],
msg='branching should result in list if `traverse_string`')
# Test re.Match as input obj
mobj = re.match(r'^0(12)(?P<group>3)(4)?$', '0123')
self.assertEqual(traverse_obj(mobj, Ellipsis), [x for x in mobj.groups() if x is not None],
msg='`...` on a `re.Match` should give its `groups()`')
self.assertEqual(traverse_obj(mobj, lambda k, _: k in (0, 2)), ['0123', '3'],
msg='function on a `re.Match` should give groupno, value starting at 0')
self.assertEqual(traverse_obj(mobj, 'group'), '3',
msg='str key on a `re.Match` should give group with that name')
self.assertEqual(traverse_obj(mobj, 2), '3',
msg='int key on a `re.Match` should give group with that name')
self.assertEqual(traverse_obj(mobj, 'gRoUp', casesense=False), '3',
msg='str key on a `re.Match` should respect casesense')
self.assertEqual(traverse_obj(mobj, 'fail'), None,
msg='failing str key on a `re.Match` should return `default`')
self.assertEqual(traverse_obj(mobj, 'gRoUpS', casesense=False), None,
msg='failing str key on a `re.Match` should return `default`')
self.assertEqual(traverse_obj(mobj, 8), None,
msg='failing int key on a `re.Match` should return `default`')
self.assertEqual(traverse_obj(mobj, lambda k, _: k in (0, 'group')), ['0123', '3'],
msg='function on a `re.Match` should give group name as well')
# Test xml.etree.ElementTree.Element as input obj
etree = compat_etree_fromstring('''<?xml version="1.0"?>
<data>
<country name="Liechtenstein">
<rank>1</rank>
<year>2008</year>
<gdppc>141100</gdppc>
<neighbor name="Austria" direction="E"/>
<neighbor name="Switzerland" direction="W"/>
</country>
<country name="Singapore">
<rank>4</rank>
<year>2011</year>
<gdppc>59900</gdppc>
<neighbor name="Malaysia" direction="N"/>
</country>
<country name="Panama">
<rank>68</rank>
<year>2011</year>
<gdppc>13600</gdppc>
<neighbor name="Costa Rica" direction="W"/>
<neighbor name="Colombia" direction="E"/>
</country>
</data>''')
self.assertEqual(traverse_obj(etree, ''), etree,
msg='empty str key should return the element itself')
self.assertEqual(traverse_obj(etree, 'country'), list(etree),
msg='str key should return all children with that tag name')
self.assertEqual(traverse_obj(etree, Ellipsis), list(etree),
msg='`...` as key should return all children')
self.assertEqual(traverse_obj(etree, lambda _, x: x[0].text == '4'), [etree[1]],
msg='function as key should get element as value')
self.assertEqual(traverse_obj(etree, lambda i, _: i == 1), [etree[1]],
msg='function as key should get index as key')
self.assertEqual(traverse_obj(etree, 0), etree[0],
msg='int key should return the nth child')
self.assertEqual(traverse_obj(etree, './/neighbor/@name'),
['Austria', 'Switzerland', 'Malaysia', 'Costa Rica', 'Colombia'],
msg='`@<attribute>` at end of path should give that attribute')
self.assertEqual(traverse_obj(etree, '//neighbor/@fail'), [None, None, None, None, None],
msg='`@<nonexistent>` at end of path should give `None`')
self.assertEqual(traverse_obj(etree, ('//neighbor/@', 2)), {'name': 'Malaysia', 'direction': 'N'},
msg='`@` should give the full attribute dict')
self.assertEqual(traverse_obj(etree, '//year/text()'), ['2008', '2011', '2011'],
msg='`text()` at end of path should give the inner text')
self.assertEqual(traverse_obj(etree, '//*[@direction]/@direction'), ['E', 'W', 'N', 'W', 'E'],
msg='full python xpath features should be supported')
self.assertEqual(traverse_obj(etree, (0, '@name')), 'Liechtenstein',
msg='special transformations should act on current element')
self.assertEqual(traverse_obj(etree, ('country', 0, Ellipsis, 'text()', T(int_or_none))), [1, 2008, 141100],
msg='special transformations should act on current element')
def test_traversal_unbranching(self):
self.assertEqual(traverse_obj(_TEST_DATA, [(100, 1.2), all]), [100, 1.2],
msg='`all` should give all results as list')
self.assertEqual(traverse_obj(_TEST_DATA, [(100, 1.2), any]), 100,
msg='`any` should give the first result')
self.assertEqual(traverse_obj(_TEST_DATA, [100, all]), [100],
msg='`all` should give list if non branching')
self.assertEqual(traverse_obj(_TEST_DATA, [100, any]), 100,
msg='`any` should give single item if non branching')
self.assertEqual(traverse_obj(_TEST_DATA, [('dict', 'None', 100), all]), [100],
msg='`all` should filter `None` and empty dict')
self.assertEqual(traverse_obj(_TEST_DATA, [('dict', 'None', 100), any]), 100,
msg='`any` should filter `None` and empty dict')
self.assertEqual(traverse_obj(_TEST_DATA, [{
'all': [('dict', 'None', 100, 1.2), all],
'any': [('dict', 'None', 100, 1.2), any],
}]), {'all': [100, 1.2], 'any': 100},
msg='`all`/`any` should apply to each dict path separately')
self.assertEqual(traverse_obj(_TEST_DATA, [{
'all': [('dict', 'None', 100, 1.2), all],
'any': [('dict', 'None', 100, 1.2), any],
}], get_all=False), {'all': [100, 1.2], 'any': 100},
msg='`all`/`any` should apply to dict regardless of `get_all`')
self.assertIs(traverse_obj(_TEST_DATA, [('dict', 'None', 100, 1.2), all, T(float)]), None,
msg='`all` should reset branching status')
self.assertIs(traverse_obj(_TEST_DATA, [('dict', 'None', 100, 1.2), any, T(float)]), None,
msg='`any` should reset branching status')
self.assertEqual(traverse_obj(_TEST_DATA, [('dict', 'None', 100, 1.2), all, Ellipsis, T(float)]), [1.2],
msg='`all` should allow further branching')
self.assertEqual(traverse_obj(_TEST_DATA, [('dict', 'None', 'urls', 'data'), any, Ellipsis, 'index']), [0, 1],
msg='`any` should allow further branching')
def test_traversal_morsel(self):
values = {
'expires': 'a',
'path': 'b',
'comment': 'c',
'domain': 'd',
'max-age': 'e',
'secure': 'f',
'httponly': 'g',
'version': 'h',
'samesite': 'i',
}
# SameSite added in Py3.8, breaks .update for 3.5-3.7
if sys.version_info < (3, 8):
del values['samesite']
morsel = compat_http_cookies.Morsel()
morsel.set(str('item_key'), 'item_value', 'coded_value')
morsel.update(values)
values['key'] = str('item_key')
values['value'] = 'item_value'
values = dict((str(k), v) for k, v in values.items())
# make test pass even without ordered dict
value_set = set(values.values())
for key, value in values.items():
self.assertEqual(traverse_obj(morsel, key), value,
msg='Morsel should provide access to all values')
self.assertEqual(set(traverse_obj(morsel, Ellipsis)), value_set,
msg='`...` should yield all values')
self.assertEqual(set(traverse_obj(morsel, lambda k, v: True)), value_set,
msg='function key should yield all values')
self.assertIs(traverse_obj(morsel, [(None,), any]), morsel,
msg='Morsel should not be implicitly changed to dict on usage')
def test_get_first(self):
self.assertEqual(get_first([{'a': None}, {'a': 'spam'}], 'a'), 'spam')
def test_dict_get(self):
FALSE_VALUES = {
'none': None,
'false': False,
'zero': 0,
'empty_string': '',
'empty_list': [],
}
d = FALSE_VALUES.copy()
d['a'] = 42
self.assertEqual(dict_get(d, 'a'), 42)
self.assertEqual(dict_get(d, 'b'), None)
self.assertEqual(dict_get(d, 'b', 42), 42)
self.assertEqual(dict_get(d, ('a', )), 42)
self.assertEqual(dict_get(d, ('b', 'a', )), 42)
self.assertEqual(dict_get(d, ('b', 'c', 'a', 'd', )), 42)
self.assertEqual(dict_get(d, ('b', 'c', )), None)
self.assertEqual(dict_get(d, ('b', 'c', ), 42), 42)
for key, false_value in FALSE_VALUES.items():
self.assertEqual(dict_get(d, ('b', 'c', key, )), None)
self.assertEqual(dict_get(d, ('b', 'c', key, ), skip_false_values=False), false_value)
if __name__ == '__main__':
unittest.main()

View File

@ -2,19 +2,21 @@ from __future__ import unicode_literals
# Allow direct execution
import os
import re
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import io
import re
dirn = os.path.dirname
rootDir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
rootDir = dirn(dirn(os.path.abspath(__file__)))
sys.path.insert(0, rootDir)
IGNORED_FILES = [
'setup.py', # http://bugs.python.org/issue13943
'conf.py',
'buildserver.py',
'get-pip.py',
]
IGNORED_DIRS = [
@ -23,6 +25,7 @@ IGNORED_DIRS = [
]
from test.helper import assertRegexpMatches
from youtube_dl.compat import compat_open as open
class TestUnicodeLiterals(unittest.TestCase):
@ -40,7 +43,7 @@ class TestUnicodeLiterals(unittest.TestCase):
continue
fn = os.path.join(dirpath, basename)
with io.open(fn, encoding='utf-8') as inf:
with open(fn, encoding='utf-8') as inf:
code = inf.read()
if "'" not in code and '"' not in code:

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,71 @@
#!/usr/bin/env python
# coding: utf-8
from __future__ import unicode_literals
import unittest
import sys
import os
import subprocess
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
rootDir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
class TestVerboseOutput(unittest.TestCase):
def test_private_info_arg(self):
outp = subprocess.Popen(
[
sys.executable, 'youtube_dl/__main__.py', '-v',
'--username', 'johnsmith@gmail.com',
'--password', 'secret',
], cwd=rootDir, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
sout, serr = outp.communicate()
self.assertTrue(b'--username' in serr)
self.assertTrue(b'johnsmith' not in serr)
self.assertTrue(b'--password' in serr)
self.assertTrue(b'secret' not in serr)
def test_private_info_shortarg(self):
outp = subprocess.Popen(
[
sys.executable, 'youtube_dl/__main__.py', '-v',
'-u', 'johnsmith@gmail.com',
'-p', 'secret',
], cwd=rootDir, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
sout, serr = outp.communicate()
self.assertTrue(b'-u' in serr)
self.assertTrue(b'johnsmith' not in serr)
self.assertTrue(b'-p' in serr)
self.assertTrue(b'secret' not in serr)
def test_private_info_eq(self):
outp = subprocess.Popen(
[
sys.executable, 'youtube_dl/__main__.py', '-v',
'--username=johnsmith@gmail.com',
'--password=secret',
], cwd=rootDir, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
sout, serr = outp.communicate()
self.assertTrue(b'--username' in serr)
self.assertTrue(b'johnsmith' not in serr)
self.assertTrue(b'--password' in serr)
self.assertTrue(b'secret' not in serr)
def test_private_info_shortarg_eq(self):
outp = subprocess.Popen(
[
sys.executable, 'youtube_dl/__main__.py', '-v',
'-u=johnsmith@gmail.com',
'-p=secret',
], cwd=rootDir, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
sout, serr = outp.communicate()
self.assertTrue(b'-u' in serr)
self.assertTrue(b'johnsmith' not in serr)
self.assertTrue(b'-p' in serr)
self.assertTrue(b'secret' not in serr)
if __name__ == '__main__':
unittest.main()

View File

@ -11,12 +11,11 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from test.helper import get_params, try_rm
import io
import xml.etree.ElementTree
import youtube_dl.YoutubeDL
import youtube_dl.extractor
from youtube_dl.compat import compat_open as open
class YoutubeDL(youtube_dl.YoutubeDL):
@ -24,6 +23,7 @@ class YoutubeDL(youtube_dl.YoutubeDL):
super(YoutubeDL, self).__init__(*args, **kwargs)
self.to_stderr = self.to_screen
params = get_params({
'writeannotations': True,
'skip_download': True,
@ -50,7 +50,7 @@ class TestAnnotations(unittest.TestCase):
ydl.download([TEST_ID])
self.assertTrue(os.path.exists(ANNOTATIONS_FILE))
annoxml = None
with io.open(ANNOTATIONS_FILE, 'r', encoding='utf-8') as annof:
with open(ANNOTATIONS_FILE, 'r', encoding='utf-8') as annof:
annoxml = xml.etree.ElementTree.parse(annof)
self.assertTrue(annoxml is not None, 'Failed to parse annotations XML')
root = annoxml.getroot()
@ -74,5 +74,6 @@ class TestAnnotations(unittest.TestCase):
def tearDown(self):
try_rm(ANNOTATIONS_FILE)
if __name__ == '__main__':
unittest.main()

View File

@ -1,4 +1,5 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
# Allow direct execution
@ -9,10 +10,10 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from test.helper import FakeYDL
from youtube_dl.extractor import (
YoutubePlaylistIE,
YoutubeIE,
YoutubePlaylistIE,
YoutubeTabIE,
)
@ -24,47 +25,41 @@ class TestYoutubeLists(unittest.TestCase):
def test_youtube_playlist_noplaylist(self):
dl = FakeYDL()
dl.params['noplaylist'] = True
dl.params['format'] = 'best'
ie = YoutubePlaylistIE(dl)
result = ie.extract('https://www.youtube.com/watch?v=FXxLjLQi3Fg&list=PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re')
self.assertEqual(result['_type'], 'url')
result = dl.extract_info(result['url'], download=False, ie_key=result.get('ie_key'), process=False)
self.assertEqual(YoutubeIE().extract_id(result['url']), 'FXxLjLQi3Fg')
def test_youtube_course(self):
dl = FakeYDL()
ie = YoutubePlaylistIE(dl)
# TODO find a > 100 (paginating?) videos course
result = ie.extract('https://www.youtube.com/course?list=ECUl4u3cNGP61MdtwGTqZA0MreSaDybji8')
entries = list(result['entries'])
self.assertEqual(YoutubeIE().extract_id(entries[0]['url']), 'j9WZyLZCBzs')
self.assertEqual(len(entries), 25)
self.assertEqual(YoutubeIE().extract_id(entries[-1]['url']), 'rYefUsYuEp0')
def test_youtube_mix(self):
dl = FakeYDL()
ie = YoutubePlaylistIE(dl)
result = ie.extract('https://www.youtube.com/watch?v=W01L70IGBgE&index=2&list=RDOQpdSVF_k_w')
entries = result['entries']
self.assertTrue(len(entries) >= 20)
dl.params['format'] = 'best'
ie = YoutubeTabIE(dl)
result = dl.extract_info('https://www.youtube.com/watch?v=tyITL_exICo&list=RDCLAK5uy_kLWIr9gv1XLlPbaDS965-Db4TrBoUTxQ8',
download=False, ie_key=ie.ie_key(), process=True)
entries = (result or {}).get('entries', [{'id': 'not_found', }])
self.assertTrue(len(entries) >= 25)
original_video = entries[0]
self.assertEqual(original_video['id'], 'OQpdSVF_k_w')
self.assertEqual(original_video['id'], 'tyITL_exICo')
def test_youtube_toptracks(self):
print('Skipping: The playlist page gives error 500')
return
dl = FakeYDL()
ie = YoutubePlaylistIE(dl)
result = ie.extract('https://www.youtube.com/playlist?list=MCUS')
entries = result['entries']
self.assertEqual(len(entries), 100)
def test_youtube_flat_playlist_titles(self):
def test_youtube_flat_playlist_extraction(self):
dl = FakeYDL()
dl.params['extract_flat'] = True
ie = YoutubePlaylistIE(dl)
result = ie.extract('https://www.youtube.com/playlist?list=PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re')
ie = YoutubeTabIE(dl)
result = ie.extract('https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc')
self.assertIsPlaylist(result)
for entry in result['entries']:
self.assertTrue(entry.get('title'))
entries = list(result['entries'])
self.assertTrue(len(entries) == 1)
video = entries[0]
self.assertEqual(video['_type'], 'url')
self.assertEqual(video['ie_key'], 'Youtube')
self.assertEqual(video['id'], 'BaW_jenozKc')
self.assertEqual(video['url'], 'BaW_jenozKc')
self.assertEqual(video['title'], 'youtube-dl test video "\'/\\ä↭𝕐')
self.assertEqual(video['duration'], 10)
self.assertEqual(video['uploader'], 'Philipp Hagemeister')
if __name__ == '__main__':
unittest.main()

26
test/test_youtube_misc.py Normal file
View File

@ -0,0 +1,26 @@
#!/usr/bin/env python
from __future__ import unicode_literals
# Allow direct execution
import os
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from youtube_dl.extractor import YoutubeIE
class TestYoutubeMisc(unittest.TestCase):
def test_youtube_extract(self):
assertExtractId = lambda url, id: self.assertEqual(YoutubeIE.extract_id(url), id)
assertExtractId('http://www.youtube.com/watch?&v=BaW_jenozKc', 'BaW_jenozKc')
assertExtractId('https://www.youtube.com/watch?&v=BaW_jenozKc', 'BaW_jenozKc')
assertExtractId('https://www.youtube.com/watch?feature=player_embedded&v=BaW_jenozKc', 'BaW_jenozKc')
assertExtractId('https://www.youtube.com/watch_popup?v=BaW_jenozKc', 'BaW_jenozKc')
assertExtractId('http://www.youtube.com/watch?v=BaW_jenozKcsharePLED17F32AD9753930', 'BaW_jenozKc')
assertExtractId('BaW_jenozKc', 'BaW_jenozKc')
if __name__ == '__main__':
unittest.main()

View File

@ -1,4 +1,5 @@
#!/usr/bin/env python
# coding: utf-8
from __future__ import unicode_literals
@ -8,114 +9,438 @@ import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import io
import re
import string
from youtube_dl.compat import (
compat_contextlib_suppress,
compat_open as open,
compat_str,
compat_urlretrieve,
)
from test.helper import FakeYDL
from youtube_dl.extractor import YoutubeIE
from youtube_dl.compat import compat_str, compat_urlretrieve
from youtube_dl.jsinterp import JSInterpreter
_TESTS = [
_SIG_TESTS = [
(
'https://s.ytimg.com/yts/jsbin/html5player-vflHOr_nV.js',
'js',
86,
'>=<;:/.-[+*)(\'&%$#"!ZYX0VUTSRQPONMLKJIHGFEDCBA\\yxwvutsrqponmlkjihgfedcba987654321',
),
(
'https://s.ytimg.com/yts/jsbin/html5player-vfldJ8xgI.js',
'js',
85,
'3456789a0cdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRS[UVWXYZ!"#$%&\'()*+,-./:;<=>?@',
),
(
'https://s.ytimg.com/yts/jsbin/html5player-vfle-mVwz.js',
'js',
90,
']\\[@?>=<;:/.-,+*)(\'&%$#"hZYXWVUTSRQPONMLKJIHGFEDCBAzyxwvutsrqponmlkjiagfedcb39876',
),
(
'https://s.ytimg.com/yts/jsbin/html5player-en_US-vfl0Cbn9e.js',
'js',
84,
'O1I3456789abcde0ghijklmnopqrstuvwxyzABCDEFGHfJKLMN2PQRSTUVW@YZ!"#$%&\'()*+,-./:;<=',
),
(
'https://s.ytimg.com/yts/jsbin/html5player-en_US-vflXGBaUN.js',
'js',
'2ACFC7A61CA478CD21425E5A57EBD73DDC78E22A.2094302436B2D377D14A3BBA23022D023B8BC25AA',
'A52CB8B320D22032ABB3A41D773D2B6342034902.A22E87CDD37DBE75A5E52412DC874AC16A7CFCA2',
),
(
'https://s.ytimg.com/yts/jsbin/html5player-en_US-vflBb0OQx.js',
'js',
84,
'123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQ0STUVWXYZ!"#$%&\'()*+,@./:;<=>'
'123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQ0STUVWXYZ!"#$%&\'()*+,@./:;<=>',
),
(
'https://s.ytimg.com/yts/jsbin/html5player-en_US-vfl9FYC6l.js',
'js',
83,
'123456789abcdefghijklmnopqr0tuvwxyzABCDETGHIJKLMNOPQRS>UVWXYZ!"#$%&\'()*+,-./:;<=F'
'123456789abcdefghijklmnopqr0tuvwxyzABCDETGHIJKLMNOPQRS>UVWXYZ!"#$%&\'()*+,-./:;<=F',
),
(
'https://s.ytimg.com/yts/jsbin/html5player-en_US-vflCGk6yw/html5player.js',
'js',
'4646B5181C6C3020DF1D9C7FCFEA.AD80ABF70C39BD369CCCAE780AFBB98FA6B6CB42766249D9488C288',
'82C8849D94266724DC6B6AF89BBFA087EACCD963.B93C07FBA084ACAEFCF7C9D1FD0203C6C1815B6B'
'82C8849D94266724DC6B6AF89BBFA087EACCD963.B93C07FBA084ACAEFCF7C9D1FD0203C6C1815B6B',
),
(
'https://s.ytimg.com/yts/jsbin/html5player-en_US-vflKjOTVq/html5player.js',
'js',
'312AA52209E3623129A412D56A40F11CB0AF14AE.3EE09501CB14E3BCDC3B2AE808BF3F1D14E7FBF12',
'112AA5220913623229A412D56A40F11CB0AF14AE.3EE0950FCB14EEBCDC3B2AE808BF331D14E7FBF3',
)
),
(
'https://www.youtube.com/s/player/6ed0d907/player_ias.vflset/en_US/base.js',
'2aq0aqSyOoJXtK73m-uME_jv7-pT15gOFC02RFkGMqWpzEICs69VdbwQ0LDp1v7j8xx92efCJlYFYb1sUkkBSPOlPmXgIARw8JQ0qOAOAA',
'AOq0QJ8wRAIgXmPlOPSBkkUs1bYFYlJCfe29xx8j7v1pDL2QwbdV96sCIEzpWqMGkFR20CFOg51Tp-7vj_EMu-m37KtXJoOySqa0',
),
(
'https://www.youtube.com/s/player/3bb1f723/player_ias.vflset/en_US/base.js',
'2aq0aqSyOoJXtK73m-uME_jv7-pT15gOFC02RFkGMqWpzEICs69VdbwQ0LDp1v7j8xx92efCJlYFYb1sUkkBSPOlPmXgIARw8JQ0qOAOAA',
'MyOSJXtKI3m-uME_jv7-pT12gOFC02RFkGoqWpzE0Cs69VdbwQ0LDp1v7j8xx92efCJlYFYb1sUkkBSPOlPmXgIARw8JQ0qOAOAA',
),
(
'https://www.youtube.com/s/player/2f1832d2/player_ias.vflset/en_US/base.js',
'2aq0aqSyOoJXtK73m-uME_jv7-pT15gOFC02RFkGMqWpzEICs69VdbwQ0LDp1v7j8xx92efCJlYFYb1sUkkBSPOlPmXgIARw8JQ0qOAOAA',
'0QJ8wRAIgXmPlOPSBkkUs1bYFYlJCfe29xxAj7v1pDL0QwbdV96sCIEzpWqMGkFR20CFOg51Tp-7vj_EMu-m37KtXJ2OySqa0q',
),
(
'https://www.youtube.com/s/player/643afba4/tv-player-ias.vflset/tv-player-ias.js',
'2aq0aqSyOoJXtK73m-uME_jv7-pT15gOFC02RFkGMqWpzEICs69VdbwQ0LDp1v7j8xx92efCJlYFYb1sUkkBSPOlPmXgIARw8JQ0qOAOAA',
'AAOAOq0QJ8wRAIgXmPlOPSBkkUs1bYFYlJCfe29xx8j7vgpDL0QwbdV06sCIEzpWqMGkFR20CFOS21Tp-7vj_EMu-m37KtXJoOy1',
),
(
'https://www.youtube.com/s/player/363db69b/player_ias.vflset/en_US/base.js',
'2aq0aqSyOoJXtK73m-uME_jv7-pT15gOFC02RFkGMqWpzEICs69VdbwQ0LDp1v7j8xx92efCJlYFYb1sUkkBSPOlPmXgIARw8JQ0qOAOAA',
'0aqSyOoJXtK73m-uME_jv7-pT15gOFC02RFkGMqWpz2ICs6EVdbwQ0LDp1v7j8xx92efCJlYFYb1sUkkBSPOlPmXgIARw8JQ0qOAOAA',
),
(
'https://www.youtube.com/s/player/363db69b/player_ias_tce.vflset/en_US/base.js',
'2aq0aqSyOoJXtK73m-uME_jv7-pT15gOFC02RFkGMqWpzEICs69VdbwQ0LDp1v7j8xx92efCJlYFYb1sUkkBSPOlPmXgIARw8JQ0qOAOAA',
'0aqSyOoJXtK73m-uME_jv7-pT15gOFC02RFkGMqWpz2ICs6EVdbwQ0LDp1v7j8xx92efCJlYFYb1sUkkBSPOlPmXgIARw8JQ0qOAOAA',
),
(
'https://www.youtube.com/s/player/4fcd6e4a/player_ias.vflset/en_US/base.js',
'2aq0aqSyOoJXtK73m-uME_jv7-pT15gOFC02RFkGMqWpzEICs69VdbwQ0LDp1v7j8xx92efCJlYFYb1sUkkBSPOlPmXgIARw8JQ0qOAOAA',
'wAOAOq0QJ8ARAIgXmPlOPSBkkUs1bYFYlJCfe29xx8q7v1pDL0QwbdV96sCIEzpWqMGkFR20CFOg51Tp-7vj_EMu-m37KtXJoOySqa0',
),
(
'https://www.youtube.com/s/player/4fcd6e4a/player_ias_tce.vflset/en_US/base.js',
'2aq0aqSyOoJXtK73m-uME_jv7-pT15gOFC02RFkGMqWpzEICs69VdbwQ0LDp1v7j8xx92efCJlYFYb1sUkkBSPOlPmXgIARw8JQ0qOAOAA',
'wAOAOq0QJ8ARAIgXmPlOPSBkkUs1bYFYlJCfe29xx8q7v1pDL0QwbdV96sCIEzpWqMGkFR20CFOg51Tp-7vj_EMu-m37KtXJoOySqa0',
),
(
'https://www.youtube.com/s/player/20830619/player_ias.vflset/en_US/base.js',
'2aq0aqSyOoJXtK73m-uME_jv7-pT15gOFC02RFkGMqWpzEICs69VdbwQ0LDp1v7j8xx92efCJlYFYb1sUkkBSPOlPmXgIARw8JQ0qOAOAA',
'7AOq0QJ8wRAIgXmPlOPSBkkAs1bYFYlJCfe29xx8jOv1pDL0Q2bdV96sCIEzpWqMGkFR20CFOg51Tp-7vj_EMu-m37KtXJoOySqa0qaw',
),
(
'https://www.youtube.com/s/player/20830619/player_ias_tce.vflset/en_US/base.js',
'2aq0aqSyOoJXtK73m-uME_jv7-pT15gOFC02RFkGMqWpzEICs69VdbwQ0LDp1v7j8xx92efCJlYFYb1sUkkBSPOlPmXgIARw8JQ0qOAOAA',
'7AOq0QJ8wRAIgXmPlOPSBkkAs1bYFYlJCfe29xx8jOv1pDL0Q2bdV96sCIEzpWqMGkFR20CFOg51Tp-7vj_EMu-m37KtXJoOySqa0qaw',
),
(
'https://www.youtube.com/s/player/20830619/player-plasma-ias-phone-en_US.vflset/base.js',
'2aq0aqSyOoJXtK73m-uME_jv7-pT15gOFC02RFkGMqWpzEICs69VdbwQ0LDp1v7j8xx92efCJlYFYb1sUkkBSPOlPmXgIARw8JQ0qOAOAA',
'7AOq0QJ8wRAIgXmPlOPSBkkAs1bYFYlJCfe29xx8jOv1pDL0Q2bdV96sCIEzpWqMGkFR20CFOg51Tp-7vj_EMu-m37KtXJoOySqa0qaw',
),
(
'https://www.youtube.com/s/player/20830619/player-plasma-ias-tablet-en_US.vflset/base.js',
'2aq0aqSyOoJXtK73m-uME_jv7-pT15gOFC02RFkGMqWpzEICs69VdbwQ0LDp1v7j8xx92efCJlYFYb1sUkkBSPOlPmXgIARw8JQ0qOAOAA',
'7AOq0QJ8wRAIgXmPlOPSBkkAs1bYFYlJCfe29xx8jOv1pDL0Q2bdV96sCIEzpWqMGkFR20CFOg51Tp-7vj_EMu-m37KtXJoOySqa0qaw',
),
(
'https://www.youtube.com/s/player/8a8ac953/player_ias_tce.vflset/en_US/base.js',
'2aq0aqSyOoJXtK73m-uME_jv7-pT15gOFC02RFkGMqWpzEICs69VdbwQ0LDp1v7j8xx92efCJlYFYb1sUkkBSPOlPmXgIARw8JQ0qOAOAA',
'IAOAOq0QJ8wRAAgXmPlOPSBkkUs1bYFYlJCfe29xx8j7v1pDL0QwbdV96sCIEzpWqMGkFR20CFOg51Tp-7vj_E2u-m37KtXJoOySqa0',
),
(
'https://www.youtube.com/s/player/8a8ac953/tv-player-es6.vflset/tv-player-es6.js',
'2aq0aqSyOoJXtK73m-uME_jv7-pT15gOFC02RFkGMqWpzEICs69VdbwQ0LDp1v7j8xx92efCJlYFYb1sUkkBSPOlPmXgIARw8JQ0qOAOAA',
'IAOAOq0QJ8wRAAgXmPlOPSBkkUs1bYFYlJCfe29xx8j7v1pDL0QwbdV96sCIEzpWqMGkFR20CFOg51Tp-7vj_E2u-m37KtXJoOySqa0',
),
]
_NSIG_TESTS = [
(
'https://www.youtube.com/s/player/7862ca1f/player_ias.vflset/en_US/base.js',
'X_LCxVDjAavgE5t', 'yxJ1dM6iz5ogUg',
),
(
'https://www.youtube.com/s/player/9216d1f7/player_ias.vflset/en_US/base.js',
'SLp9F5bwjAdhE9F-', 'gWnb9IK2DJ8Q1w',
),
(
'https://www.youtube.com/s/player/f8cb7a3b/player_ias.vflset/en_US/base.js',
'oBo2h5euWy6osrUt', 'ivXHpm7qJjJN',
),
(
'https://www.youtube.com/s/player/2dfe380c/player_ias.vflset/en_US/base.js',
'oBo2h5euWy6osrUt', '3DIBbn3qdQ',
),
(
'https://www.youtube.com/s/player/f1ca6900/player_ias.vflset/en_US/base.js',
'cu3wyu6LQn2hse', 'jvxetvmlI9AN9Q',
),
(
'https://www.youtube.com/s/player/8040e515/player_ias.vflset/en_US/base.js',
'wvOFaY-yjgDuIEg5', 'HkfBFDHmgw4rsw',
),
(
'https://www.youtube.com/s/player/e06dea74/player_ias.vflset/en_US/base.js',
'AiuodmaDDYw8d3y4bf', 'ankd8eza2T6Qmw',
),
(
'https://www.youtube.com/s/player/5dd88d1d/player-plasma-ias-phone-en_US.vflset/base.js',
'kSxKFLeqzv_ZyHSAt', 'n8gS8oRlHOxPFA',
),
(
'https://www.youtube.com/s/player/324f67b9/player_ias.vflset/en_US/base.js',
'xdftNy7dh9QGnhW', '22qLGxrmX8F1rA',
),
(
'https://www.youtube.com/s/player/4c3f79c5/player_ias.vflset/en_US/base.js',
'TDCstCG66tEAO5pR9o', 'dbxNtZ14c-yWyw',
),
(
'https://www.youtube.com/s/player/c81bbb4a/player_ias.vflset/en_US/base.js',
'gre3EcLurNY2vqp94', 'Z9DfGxWP115WTg',
),
(
'https://www.youtube.com/s/player/1f7d5369/player_ias.vflset/en_US/base.js',
'batNX7sYqIJdkJ', 'IhOkL_zxbkOZBw',
),
(
'https://www.youtube.com/s/player/009f1d77/player_ias.vflset/en_US/base.js',
'5dwFHw8aFWQUQtffRq', 'audescmLUzI3jw',
),
(
'https://www.youtube.com/s/player/dc0c6770/player_ias.vflset/en_US/base.js',
'5EHDMgYLV6HPGk_Mu-kk', 'n9lUJLHbxUI0GQ',
),
(
'https://www.youtube.com/s/player/c2199353/player_ias.vflset/en_US/base.js',
'5EHDMgYLV6HPGk_Mu-kk', 'AD5rgS85EkrE7',
),
(
'https://www.youtube.com/s/player/113ca41c/player_ias.vflset/en_US/base.js',
'cgYl-tlYkhjT7A', 'hI7BBr2zUgcmMg',
),
(
'https://www.youtube.com/s/player/c57c113c/player_ias.vflset/en_US/base.js',
'M92UUMHa8PdvPd3wyM', '3hPqLJsiNZx7yA',
),
(
'https://www.youtube.com/s/player/5a3b6271/player_ias.vflset/en_US/base.js',
'B2j7f_UPT4rfje85Lu_e', 'm5DmNymaGQ5RdQ',
),
(
'https://www.youtube.com/s/player/7a062b77/player_ias.vflset/en_US/base.js',
'NRcE3y3mVtm_cV-W', 'VbsCYUATvqlt5w',
),
(
'https://www.youtube.com/s/player/dac945fd/player_ias.vflset/en_US/base.js',
'o8BkRxXhuYsBCWi6RplPdP', '3Lx32v_hmzTm6A',
),
(
'https://www.youtube.com/s/player/6f20102c/player_ias.vflset/en_US/base.js',
'lE8DhoDmKqnmJJ', 'pJTTX6XyJP2BYw',
),
(
'https://www.youtube.com/s/player/cfa9e7cb/player_ias.vflset/en_US/base.js',
'aCi3iElgd2kq0bxVbQ', 'QX1y8jGb2IbZ0w',
),
(
'https://www.youtube.com/s/player/8c7583ff/player_ias.vflset/en_US/base.js',
'1wWCVpRR96eAmMI87L', 'KSkWAVv1ZQxC3A',
),
(
'https://www.youtube.com/s/player/b7910ca8/player_ias.vflset/en_US/base.js',
'_hXMCwMt9qE310D', 'LoZMgkkofRMCZQ',
),
(
'https://www.youtube.com/s/player/590f65a6/player_ias.vflset/en_US/base.js',
'1tm7-g_A9zsI8_Lay_', 'xI4Vem4Put_rOg',
),
(
'https://www.youtube.com/s/player/b22ef6e7/player_ias.vflset/en_US/base.js',
'b6HcntHGkvBLk_FRf', 'kNPW6A7FyP2l8A',
),
(
'https://www.youtube.com/s/player/3400486c/player_ias.vflset/en_US/base.js',
'lL46g3XifCKUZn1Xfw', 'z767lhet6V2Skl',
),
(
'https://www.youtube.com/s/player/5604538d/player_ias.vflset/en_US/base.js',
'7X-he4jjvMx7BCX', 'sViSydX8IHtdWA',
),
(
'https://www.youtube.com/s/player/20dfca59/player_ias.vflset/en_US/base.js',
'-fLCxedkAk4LUTK2', 'O8kfRq1y1eyHGw',
),
(
'https://www.youtube.com/s/player/b12cc44b/player_ias.vflset/en_US/base.js',
'keLa5R2U00sR9SQK', 'N1OGyujjEwMnLw',
),
(
'https://www.youtube.com/s/player/3bb1f723/player_ias.vflset/en_US/base.js',
'gK15nzVyaXE9RsMP3z', 'ZFFWFLPWx9DEgQ',
),
(
'https://www.youtube.com/s/player/f8f53e1a/player_ias.vflset/en_US/base.js',
'VTQOUOv0mCIeJ7i8kZB', 'kcfD8wy0sNLyNQ',
),
(
'https://www.youtube.com/s/player/2f1832d2/player_ias.vflset/en_US/base.js',
'YWt1qdbe8SAfkoPHW5d', 'RrRjWQOJmBiP',
),
(
'https://www.youtube.com/s/player/9c6dfc4a/player_ias.vflset/en_US/base.js',
'jbu7ylIosQHyJyJV', 'uwI0ESiynAmhNg',
),
(
'https://www.youtube.com/s/player/f6e09c70/player_ias.vflset/en_US/base.js',
'W9HJZKktxuYoDTqW', 'jHbbkcaxm54',
),
(
'https://www.youtube.com/s/player/f6e09c70/player_ias_tce.vflset/en_US/base.js',
'W9HJZKktxuYoDTqW', 'jHbbkcaxm54',
),
(
'https://www.youtube.com/s/player/e7567ecf/player_ias_tce.vflset/en_US/base.js',
'Sy4aDGc0VpYRR9ew_', '5UPOT1VhoZxNLQ',
),
(
'https://www.youtube.com/s/player/d50f54ef/player_ias_tce.vflset/en_US/base.js',
'Ha7507LzRmH3Utygtj', 'XFTb2HoeOE5MHg',
),
(
'https://www.youtube.com/s/player/074a8365/player_ias_tce.vflset/en_US/base.js',
'Ha7507LzRmH3Utygtj', 'ufTsrE0IVYrkl8v',
),
(
'https://www.youtube.com/s/player/643afba4/player_ias.vflset/en_US/base.js',
'N5uAlLqm0eg1GyHO', 'dCBQOejdq5s-ww',
),
(
'https://www.youtube.com/s/player/69f581a5/tv-player-ias.vflset/tv-player-ias.js',
'-qIP447rVlTTwaZjY', 'KNcGOksBAvwqQg',
),
(
'https://www.youtube.com/s/player/643afba4/tv-player-ias.vflset/tv-player-ias.js',
'ir9-V6cdbCiyKxhr', '2PL7ZDYAALMfmA',
),
(
'https://www.youtube.com/s/player/643afba4/player_ias.vflset/en_US/base.js',
'ir9-V6cdbCiyKxhr', '2PL7ZDYAALMfmA',
),
(
'https://www.youtube.com/s/player/363db69b/player_ias.vflset/en_US/base.js',
'eWYu5d5YeY_4LyEDc', 'XJQqf-N7Xra3gg',
),
(
'https://www.youtube.com/s/player/4fcd6e4a/player_ias.vflset/en_US/base.js',
'o_L251jm8yhZkWtBW', 'lXoxI3XvToqn6A',
),
(
'https://www.youtube.com/s/player/4fcd6e4a/tv-player-ias.vflset/tv-player-ias.js',
'o_L251jm8yhZkWtBW', 'lXoxI3XvToqn6A',
),
(
'https://www.youtube.com/s/player/20830619/tv-player-ias.vflset/tv-player-ias.js',
'ir9-V6cdbCiyKxhr', '9YE85kNjZiS4',
),
(
'https://www.youtube.com/s/player/20830619/player-plasma-ias-phone-en_US.vflset/base.js',
'ir9-V6cdbCiyKxhr', '9YE85kNjZiS4',
),
(
'https://www.youtube.com/s/player/20830619/player-plasma-ias-tablet-en_US.vflset/base.js',
'ir9-V6cdbCiyKxhr', '9YE85kNjZiS4',
),
(
'https://www.youtube.com/s/player/8a8ac953/player_ias_tce.vflset/en_US/base.js',
'MiBYeXx_vRREbiCCmh', 'RtZYMVvmkE0JE',
),
(
'https://www.youtube.com/s/player/8a8ac953/tv-player-es6.vflset/tv-player-es6.js',
'MiBYeXx_vRREbiCCmh', 'RtZYMVvmkE0JE',
),
(
'https://www.youtube.com/s/player/aa3fc80b/player_ias.vflset/en_US/base.js',
'0qY9dal2uzOnOGwa-48hha', 'VSh1KDfQMk-eag',
),
]
class TestPlayerInfo(unittest.TestCase):
def test_youtube_extract_player_info(self):
PLAYER_URLS = (
('https://www.youtube.com/s/player/4c3f79c5/player_ias.vflset/en_US/base.js', '4c3f79c5'),
('https://www.youtube.com/s/player/64dddad9/player_ias.vflset/en_US/base.js', '64dddad9'),
('https://www.youtube.com/s/player/64dddad9/player_ias.vflset/fr_FR/base.js', '64dddad9'),
('https://www.youtube.com/s/player/64dddad9/player-plasma-ias-phone-en_US.vflset/base.js', '64dddad9'),
('https://www.youtube.com/s/player/64dddad9/player-plasma-ias-phone-de_DE.vflset/base.js', '64dddad9'),
('https://www.youtube.com/s/player/64dddad9/player-plasma-ias-tablet-en_US.vflset/base.js', '64dddad9'),
('https://www.youtube.com/s/player/e7567ecf/player_ias_tce.vflset/en_US/base.js', 'e7567ecf'),
('https://www.youtube.com/s/player/643afba4/tv-player-ias.vflset/tv-player-ias.js', '643afba4'),
# obsolete
('https://www.youtube.com/yts/jsbin/player_ias-vfle4-e03/en_US/base.js', 'vfle4-e03'),
('https://www.youtube.com/yts/jsbin/player_ias-vfl49f_g4/en_US/base.js', 'vfl49f_g4'),
('https://www.youtube.com/yts/jsbin/player_ias-vflCPQUIL/en_US/base.js', 'vflCPQUIL'),
('https://www.youtube.com/yts/jsbin/player-vflzQZbt7/en_US/base.js', 'vflzQZbt7'),
('https://www.youtube.com/yts/jsbin/player-en_US-vflaxXRn1/base.js', 'vflaxXRn1'),
('https://s.ytimg.com/yts/jsbin/html5player-en_US-vflXGBaUN.js', 'vflXGBaUN'),
('https://s.ytimg.com/yts/jsbin/html5player-en_US-vflKjOTVq/html5player.js', 'vflKjOTVq'),
)
ie = YoutubeIE(FakeYDL({'cachedir': False}))
for player_url, expected_player_id in PLAYER_URLS:
player_id = ie._extract_player_info(player_url)
self.assertEqual(player_id, expected_player_id)
class TestSignature(unittest.TestCase):
def setUp(self):
TEST_DIR = os.path.dirname(os.path.abspath(__file__))
self.TESTDATA_DIR = os.path.join(TEST_DIR, 'testdata')
self.TESTDATA_DIR = os.path.join(TEST_DIR, 'testdata/sigs')
if not os.path.exists(self.TESTDATA_DIR):
os.mkdir(self.TESTDATA_DIR)
def tearDown(self):
with compat_contextlib_suppress(OSError):
for f in os.listdir(self.TESTDATA_DIR):
os.remove(f)
def make_tfunc(url, stype, sig_input, expected_sig):
m = re.match(r'.*-([a-zA-Z0-9_-]+)(?:/watch_as3|/html5player)?\.[a-z]+$', url)
assert m, '%r should follow URL format' % url
test_id = m.group(1)
def test_func(self):
basename = 'player-%s.%s' % (test_id, stype)
fn = os.path.join(self.TESTDATA_DIR, basename)
def t_factory(name, sig_func, url_pattern):
def make_tfunc(url, sig_input, expected_sig):
m = url_pattern.match(url)
assert m, '{0!r} should follow URL format'.format(url)
test_id = re.sub(r'[/.-]', '_', m.group('id') or m.group('compat_id'))
if not os.path.exists(fn):
compat_urlretrieve(url, fn)
def test_func(self):
basename = 'player-{0}.js'.format(test_id)
fn = os.path.join(self.TESTDATA_DIR, basename)
ydl = FakeYDL()
ie = YoutubeIE(ydl)
if stype == 'js':
with io.open(fn, encoding='utf-8') as testf:
if not os.path.exists(fn):
compat_urlretrieve(url, fn)
with open(fn, encoding='utf-8') as testf:
jscode = testf.read()
func = ie._parse_sig_js(jscode)
else:
assert stype == 'swf'
with open(fn, 'rb') as testf:
swfcode = testf.read()
func = ie._parse_sig_swf(swfcode)
src_sig = (
compat_str(string.printable[:sig_input])
if isinstance(sig_input, int) else sig_input)
got_sig = func(src_sig)
self.assertEqual(got_sig, expected_sig)
self.assertEqual(sig_func(jscode, sig_input), expected_sig)
test_func.__name__ = str('test_signature_' + stype + '_' + test_id)
setattr(TestSignature, test_func.__name__, test_func)
test_func.__name__ = str('test_{0}_js_{1}'.format(name, test_id))
setattr(TestSignature, test_func.__name__, test_func)
return make_tfunc
for test_spec in _TESTS:
make_tfunc(*test_spec)
def signature(jscode, sig_input):
func = YoutubeIE(FakeYDL({'cachedir': False}))._parse_sig_js(jscode)
src_sig = (
compat_str(string.printable[:sig_input])
if isinstance(sig_input, int) else sig_input)
return func(src_sig)
def n_sig(jscode, sig_input):
ie = YoutubeIE(FakeYDL({'cachedir': False}))
jsi = JSInterpreter(jscode)
jsi, _, func_code = ie._extract_n_function_code_jsi(sig_input, jsi)
return ie._extract_n_function_from_code(jsi, func_code)(sig_input)
make_sig_test = t_factory(
'signature', signature,
re.compile(r'''(?x)
.+/(?P<h5>html5)?player(?(h5)(?:-en_US)?-|/)(?P<id>[a-zA-Z0-9/._-]+)
(?(h5)/(?:watch_as3|html5player))?\.js$
'''))
for test_spec in _SIG_TESTS:
make_sig_test(*test_spec)
make_nsig_test = t_factory(
'nsig', n_sig, re.compile(r'.+/player/(?P<id>[a-zA-Z0-9_/.-]+)\.js$'))
for test_spec in _NSIG_TESTS:
make_nsig_test(*test_spec)
if __name__ == '__main__':

View File

@ -0,0 +1,6 @@
# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This is a generated file! Do not edit.
#HttpOnly_www.foobar.foobar FALSE / TRUE 2147483647 HTTPONLY_COOKIE HTTPONLY_COOKIE_VALUE
www.foobar.foobar FALSE / TRUE 2147483647 JS_ACCESSIBLE_COOKIE JS_ACCESSIBLE_COOKIE_VALUE

View File

@ -0,0 +1,9 @@
# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This is a generated file! Do not edit.
# Cookie file entry with invalid number of fields - 6 instead of 7
www.foobar.foobar FALSE / FALSE 0 COOKIE
# Cookie file entry with invalid expires at
www.foobar.foobar FALSE / FALSE 1.7976931348623157e+308 COOKIE VALUE

View File

@ -0,0 +1,6 @@
# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This is a generated file! Do not edit.
www.foobar.foobar FALSE / TRUE YoutubeDLExpiresEmpty YoutubeDLExpiresEmptyValue
www.foobar.foobar FALSE / TRUE 0 YoutubeDLExpires0 YoutubeDLExpires0Value

10
test/testdata/f4m/custom_base_url.f4m vendored Normal file
View File

@ -0,0 +1,10 @@
<?xml version="1.0" encoding="UTF-8"?>
<manifest xmlns="http://ns.adobe.com/f4m/1.0">
<streamType>recorded</streamType>
<baseURL>http://vod.livestream.com/events/0000000000673980/</baseURL>
<duration>269.293</duration>
<bootstrapInfo profile="named" id="bootstrap_1">AAAAm2Fic3QAAAAAAAAAAQAAAAPoAAAAAAAEG+0AAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAAC4BAAAAVmFmcnQAAAAAAAAD6AAAAAAEAAAAAQAAAAAAAAAAAAAXcAAAAC0AAAAAAAQHQAAAE5UAAAAuAAAAAAAEGtUAAAEYAAAAAAAAAAAAAAAAAAAAAAA=</bootstrapInfo>
<media url="b90f532f-b0f6-4f4e-8289-706d490b2fd8_2292" bootstrapInfoId="bootstrap_1" bitrate="2148" width="1280" height="720" videoCodec="avc1.4d401f" audioCodec="mp4a.40.2">
<metadata>AgAKb25NZXRhRGF0YQgAAAAIAAhkdXJhdGlvbgBAcNSwIMSbpgAFd2lkdGgAQJQAAAAAAAAABmhlaWdodABAhoAAAAAAAAAJZnJhbWVyYXRlAEA4/7DoLwW3AA12aWRlb2RhdGFyYXRlAECe1DLgjcobAAx2aWRlb2NvZGVjaWQAQBwAAAAAAAAADWF1ZGlvZGF0YXJhdGUAQGSimlvaPKQADGF1ZGlvY29kZWNpZABAJAAAAAAAAAAACQ==</metadata>
</media>
</manifest>

View File

@ -0,0 +1,14 @@
#EXTM3U
#EXT-X-VERSION:5
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Francais",DEFAULT=NO,FORCED=NO,URI="http://replayftv-pmd.francetv.fr/subtitles/2017/16/156589847-1492488987.m3u8",LANGUAGE="fra"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac",LANGUAGE="fra",NAME="Francais",DEFAULT=YES, AUTOSELECT=YES
#EXT-X-STREAM-INF:SUBTITLES="subs",AUDIO="aac",PROGRAM-ID=1,BANDWIDTH=180000,RESOLUTION=256x144,CODECS="avc1.66.30, mp4a.40.2"
http://replayftv-vh.akamaihd.net/i/streaming-adaptatif_france-dom-tom/2017/S16/J2/156589847-58f59130c1f52-,standard1,standard2,standard3,standard4,standard5,.mp4.csmil/index_0_av.m3u8?null=0
#EXT-X-STREAM-INF:SUBTITLES="subs",AUDIO="aac",PROGRAM-ID=1,BANDWIDTH=303000,RESOLUTION=320x180,CODECS="avc1.66.30, mp4a.40.2"
http://replayftv-vh.akamaihd.net/i/streaming-adaptatif_france-dom-tom/2017/S16/J2/156589847-58f59130c1f52-,standard1,standard2,standard3,standard4,standard5,.mp4.csmil/index_1_av.m3u8?null=0
#EXT-X-STREAM-INF:SUBTITLES="subs",AUDIO="aac",PROGRAM-ID=1,BANDWIDTH=575000,RESOLUTION=512x288,CODECS="avc1.66.30, mp4a.40.2"
http://replayftv-vh.akamaihd.net/i/streaming-adaptatif_france-dom-tom/2017/S16/J2/156589847-58f59130c1f52-,standard1,standard2,standard3,standard4,standard5,.mp4.csmil/index_2_av.m3u8?null=0
#EXT-X-STREAM-INF:SUBTITLES="subs",AUDIO="aac",PROGRAM-ID=1,BANDWIDTH=831000,RESOLUTION=704x396,CODECS="avc1.77.30, mp4a.40.2"

16
test/testdata/m3u8/teamcoco_11995.m3u8 vendored Normal file
View File

@ -0,0 +1,16 @@
#EXTM3U
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-0",NAME="Default",AUTOSELECT=YES,DEFAULT=YES,URI="hls/CONAN_020217_Highlight_show-audio-160k_v4.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-1",NAME="Default",AUTOSELECT=YES,DEFAULT=YES,URI="hls/CONAN_020217_Highlight_show-audio-64k_v4.m3u8"
#EXT-X-I-FRAME-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=37862000,CODECS="avc1.4d001f",URI="hls/CONAN_020217_Highlight_show-2m_iframe.m3u8"
#EXT-X-I-FRAME-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=18750000,CODECS="avc1.4d001e",URI="hls/CONAN_020217_Highlight_show-1m_iframe.m3u8"
#EXT-X-I-FRAME-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=6535000,CODECS="avc1.42001e",URI="hls/CONAN_020217_Highlight_show-400k_iframe.m3u8"
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2374000,RESOLUTION=1024x576,CODECS="avc1.4d001f,mp4a.40.2",AUDIO="audio-0"
hls/CONAN_020217_Highlight_show-2m_v4.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1205000,RESOLUTION=640x360,CODECS="avc1.4d001e,mp4a.40.2",AUDIO="audio-0"
hls/CONAN_020217_Highlight_show-1m_v4.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=522000,RESOLUTION=400x224,CODECS="avc1.42001e,mp4a.40.2",AUDIO="audio-0"
hls/CONAN_020217_Highlight_show-400k_v4.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=413000,RESOLUTION=400x224,CODECS="avc1.42001e,mp4a.40.5",AUDIO="audio-1"
hls/CONAN_020217_Highlight_show-400k_v4.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=71000,CODECS="mp4a.40.5",AUDIO="audio-1"
hls/CONAN_020217_Highlight_show-audio-64k_v4.m3u8

28
test/testdata/m3u8/ted_18923.m3u8 vendored Normal file
View File

@ -0,0 +1,28 @@
#EXTM3U
#EXT-X-VERSION:4
#EXT-X-STREAM-INF:AUDIO="600k",BANDWIDTH=1255659,PROGRAM-ID=1,CODECS="avc1.42c01e,mp4a.40.2",RESOLUTION=640x360
/videos/BorisHesser_2018S/video/600k.m3u8?nobumpers=true&uniqueId=76011e2b
#EXT-X-STREAM-INF:AUDIO="600k",BANDWIDTH=163154,PROGRAM-ID=1,CODECS="avc1.42c00c,mp4a.40.2",RESOLUTION=320x180
/videos/BorisHesser_2018S/video/64k.m3u8?nobumpers=true&uniqueId=76011e2b
#EXT-X-STREAM-INF:AUDIO="600k",BANDWIDTH=481701,PROGRAM-ID=1,CODECS="avc1.42c015,mp4a.40.2",RESOLUTION=512x288
/videos/BorisHesser_2018S/video/180k.m3u8?nobumpers=true&uniqueId=76011e2b
#EXT-X-STREAM-INF:AUDIO="600k",BANDWIDTH=769968,PROGRAM-ID=1,CODECS="avc1.42c015,mp4a.40.2",RESOLUTION=512x288
/videos/BorisHesser_2018S/video/320k.m3u8?nobumpers=true&uniqueId=76011e2b
#EXT-X-STREAM-INF:AUDIO="600k",BANDWIDTH=984037,PROGRAM-ID=1,CODECS="avc1.42c015,mp4a.40.2",RESOLUTION=512x288
/videos/BorisHesser_2018S/video/450k.m3u8?nobumpers=true&uniqueId=76011e2b
#EXT-X-STREAM-INF:AUDIO="600k",BANDWIDTH=1693925,PROGRAM-ID=1,CODECS="avc1.4d401f,mp4a.40.2",RESOLUTION=853x480
/videos/BorisHesser_2018S/video/950k.m3u8?nobumpers=true&uniqueId=76011e2b
#EXT-X-STREAM-INF:AUDIO="600k",BANDWIDTH=2462469,PROGRAM-ID=1,CODECS="avc1.640028,mp4a.40.2",RESOLUTION=1280x720
/videos/BorisHesser_2018S/video/1500k.m3u8?nobumpers=true&uniqueId=76011e2b
#EXT-X-STREAM-INF:AUDIO="600k",BANDWIDTH=68101,PROGRAM-ID=1,CODECS="mp4a.40.2",DEFAULT=YES
/videos/BorisHesser_2018S/audio/600k.m3u8?nobumpers=true&uniqueId=76011e2b
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=74298,PROGRAM-ID=1,CODECS="avc1.42c00c",RESOLUTION=320x180,URI="/videos/BorisHesser_2018S/video/64k_iframe.m3u8?nobumpers=true&uniqueId=76011e2b"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=216200,PROGRAM-ID=1,CODECS="avc1.42c015",RESOLUTION=512x288,URI="/videos/BorisHesser_2018S/video/180k_iframe.m3u8?nobumpers=true&uniqueId=76011e2b"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=304717,PROGRAM-ID=1,CODECS="avc1.42c015",RESOLUTION=512x288,URI="/videos/BorisHesser_2018S/video/320k_iframe.m3u8?nobumpers=true&uniqueId=76011e2b"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=350933,PROGRAM-ID=1,CODECS="avc1.42c015",RESOLUTION=512x288,URI="/videos/BorisHesser_2018S/video/450k_iframe.m3u8?nobumpers=true&uniqueId=76011e2b"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=495850,PROGRAM-ID=1,CODECS="avc1.42c01e",RESOLUTION=640x360,URI="/videos/BorisHesser_2018S/video/600k_iframe.m3u8?nobumpers=true&uniqueId=76011e2b"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=810750,PROGRAM-ID=1,CODECS="avc1.4d401f",RESOLUTION=853x480,URI="/videos/BorisHesser_2018S/video/950k_iframe.m3u8?nobumpers=true&uniqueId=76011e2b"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=1273700,PROGRAM-ID=1,CODECS="avc1.640028",RESOLUTION=1280x720,URI="/videos/BorisHesser_2018S/video/1500k_iframe.m3u8?nobumpers=true&uniqueId=76011e2b"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="600k",LANGUAGE="en",NAME="Audio",AUTOSELECT=YES,DEFAULT=YES,URI="/videos/BorisHesser_2018S/audio/600k.m3u8?nobumpers=true&uniqueId=76011e2b",BANDWIDTH=614400

View File

@ -0,0 +1,13 @@
#EXTM3U
#EXT-X-VERSION:4
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio",LANGUAGE="eng",NAME="English",URI="http://k.toggle.sg/fhls/p/2082311/sp/208231100/serveFlavor/entryId/0_89q6e8ku/v/2/pv/1/flavorId/0_sa2ntrdg/name/a.mp4/index.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio",LANGUAGE="und",NAME="Undefined",URI="http://k.toggle.sg/fhls/p/2082311/sp/208231100/serveFlavor/entryId/0_89q6e8ku/v/2/pv/1/flavorId/0_r7y0nitg/name/a.mp4/index.m3u8"
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=155648,RESOLUTION=320x180,AUDIO="audio"
http://k.toggle.sg/fhls/p/2082311/sp/208231100/serveFlavor/entryId/0_89q6e8ku/v/2/pv/1/flavorId/0_qlk9hlzr/name/a.mp4/index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=502784,RESOLUTION=480x270,AUDIO="audio"
http://k.toggle.sg/fhls/p/2082311/sp/208231100/serveFlavor/entryId/0_89q6e8ku/v/2/pv/1/flavorId/0_oefackmi/name/a.mp4/index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=827392,RESOLUTION=640x360,AUDIO="audio"
http://k.toggle.sg/fhls/p/2082311/sp/208231100/serveFlavor/entryId/0_89q6e8ku/v/12/pv/1/flavorId/0_vyg9pj7k/name/a.mp4/index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1396736,RESOLUTION=854x480,AUDIO="audio"
http://k.toggle.sg/fhls/p/2082311/sp/208231100/serveFlavor/entryId/0_89q6e8ku/v/12/pv/1/flavorId/0_50n4psvx/name/a.mp4/index.m3u8

20
test/testdata/m3u8/twitch_vod.m3u8 vendored Normal file
View File

@ -0,0 +1,20 @@
#EXTM3U
#EXT-X-TWITCH-INFO:ORIGIN="s3",CLUSTER="edgecast_vod",REGION="EU",MANIFEST-CLUSTER="edgecast_vod",USER-IP="109.171.17.81"
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="chunked",NAME="Source",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=3214134,CODECS="avc1.100.31,mp4a.40.2",RESOLUTION="1280x720",VIDEO="chunked"
https://vod.edgecast.hls.ttvnw.net/e5da31ab49_riotgames_15001215120_261543898/chunked/index-muted-HM49I092CC.m3u8
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="high",NAME="High",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1603789,CODECS="avc1.42C01F,mp4a.40.2",RESOLUTION="1280x720",VIDEO="high"
https://vod.edgecast.hls.ttvnw.net/e5da31ab49_riotgames_15001215120_261543898/high/index-muted-HM49I092CC.m3u8
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="medium",NAME="Medium",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=893387,CODECS="avc1.42C01E,mp4a.40.2",RESOLUTION="852x480",VIDEO="medium"
https://vod.edgecast.hls.ttvnw.net/e5da31ab49_riotgames_15001215120_261543898/medium/index-muted-HM49I092CC.m3u8
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="low",NAME="Low",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=628347,CODECS="avc1.42C01E,mp4a.40.2",RESOLUTION="640x360",VIDEO="low"
https://vod.edgecast.hls.ttvnw.net/e5da31ab49_riotgames_15001215120_261543898/low/index-muted-HM49I092CC.m3u8
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="mobile",NAME="Mobile",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=280474,CODECS="avc1.42C00D,mp4a.40.2",RESOLUTION="400x226",VIDEO="mobile"
https://vod.edgecast.hls.ttvnw.net/e5da31ab49_riotgames_15001215120_261543898/mobile/index-muted-HM49I092CC.m3u8
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="audio_only",NAME="Audio Only",AUTOSELECT=NO,DEFAULT=NO
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=182725,CODECS="mp4a.40.2",VIDEO="audio_only"
https://vod.edgecast.hls.ttvnw.net/e5da31ab49_riotgames_15001215120_261543898/audio_only/index-muted-HM49I092CC.m3u8

10
test/testdata/m3u8/vidio.m3u8 vendored Normal file
View File

@ -0,0 +1,10 @@
#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=300000,RESOLUTION=480x270,NAME="270p 3G"
https://cdn1-a.production.vidio.static6.com/uploads/165683/dj_ambred-4383-b300.mp4.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=600000,RESOLUTION=640x360,NAME="360p SD"
https://cdn1-a.production.vidio.static6.com/uploads/165683/dj_ambred-4383-b600.mp4.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1200000,RESOLUTION=1280x720,NAME="720p HD"
https://cdn1-a.production.vidio.static6.com/uploads/165683/dj_ambred-4383-b1200.mp4.m3u8

18
test/testdata/mpd/float_duration.mpd vendored Normal file
View File

@ -0,0 +1,18 @@
<?xml version="1.0" encoding="UTF-8"?>
<MPD xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:mpeg:dash:schema:mpd:2011" type="static" minBufferTime="PT2S" profiles="urn:mpeg:dash:profile:isoff-on-demand:2011" mediaPresentationDuration="PT6014S">
<Period bitstreamSwitching="true">
<AdaptationSet mimeType="audio/mp4" codecs="mp4a.40.2" startWithSAP="1" segmentAlignment="true">
<SegmentTemplate timescale="1000000" presentationTimeOffset="0" initialization="ai_$RepresentationID$.mp4d" media="a_$RepresentationID$_$Number$.mp4d" duration="2000000.0" startNumber="0"></SegmentTemplate>
<Representation id="318597" bandwidth="61587"></Representation>
</AdaptationSet>
<AdaptationSet mimeType="video/mp4" startWithSAP="1" segmentAlignment="true">
<SegmentTemplate timescale="1000000" presentationTimeOffset="0" initialization="vi_$RepresentationID$.mp4d" media="v_$RepresentationID$_$Number$.mp4d" duration="2000000.0" startNumber="0"></SegmentTemplate>
<Representation id="318597" codecs="avc1.42001f" width="340" height="192" bandwidth="318597"></Representation>
<Representation id="638590" codecs="avc1.42001f" width="512" height="288" bandwidth="638590"></Representation>
<Representation id="1022565" codecs="avc1.4d001f" width="688" height="384" bandwidth="1022565"></Representation>
<Representation id="2046506" codecs="avc1.4d001f" width="1024" height="576" bandwidth="2046506"></Representation>
<Representation id="3998017" codecs="avc1.640029" width="1280" height="720" bandwidth="3998017"></Representation>
<Representation id="5997485" codecs="avc1.640032" width="1920" height="1080" bandwidth="5997485"></Representation>
</AdaptationSet>
</Period>
</MPD>

35
test/testdata/mpd/range_only.mpd vendored Normal file
View File

@ -0,0 +1,35 @@
<?xml version="1.0"?>
<!-- MPD file Generated with GPAC version 1.0.1-revrelease at 2021-11-27T20:53:11.690Z -->
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" minBufferTime="PT1.500S" type="static" mediaPresentationDuration="PT0H0M30.196S" maxSegmentDuration="PT0H0M10.027S" profiles="urn:mpeg:dash:profile:full:2011">
<ProgramInformation moreInformationURL="http://gpac.io">
<Title>manifest.mpd generated by GPAC</Title>
</ProgramInformation>
<Period duration="PT0H0M30.196S">
<AdaptationSet segmentAlignment="true" maxWidth="768" maxHeight="432" maxFrameRate="30000/1001" par="16:9" lang="und" startWithSAP="1">
<Representation id="1" mimeType="video/mp4" codecs="avc1.4D401E" width="768" height="432" frameRate="30000/1001" sar="1:1" bandwidth="526987">
<BaseURL>video_dashinit.mp4</BaseURL>
<SegmentList timescale="90000" duration="900000">
<Initialization range="0-881"/>
<SegmentURL mediaRange="882-876094" indexRange="882-925"/>
<SegmentURL mediaRange="876095-1466732" indexRange="876095-876138"/>
<SegmentURL mediaRange="1466733-1953615" indexRange="1466733-1466776"/>
<SegmentURL mediaRange="1953616-1994211" indexRange="1953616-1953659"/>
</SegmentList>
</Representation>
</AdaptationSet>
<AdaptationSet segmentAlignment="true" lang="und" startWithSAP="1">
<Representation id="2" mimeType="audio/mp4" codecs="mp4a.40.2" audioSamplingRate="48000" bandwidth="98096">
<AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
<BaseURL>audio_dashinit.mp4</BaseURL>
<SegmentList timescale="48000" duration="480000">
<Initialization range="0-752"/>
<SegmentURL mediaRange="753-124129" indexRange="753-796"/>
<SegmentURL mediaRange="124130-250544" indexRange="124130-124173"/>
<SegmentURL mediaRange="250545-374929" indexRange="250545-250588"/>
</SegmentList>
</Representation>
</AdaptationSet>
</Period>
</MPD>

351
test/testdata/mpd/subtitles.mpd vendored Normal file
View File

@ -0,0 +1,351 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Created with Unified Streaming Platform (version=1.10.18-20255) -->
<MPD
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="urn:mpeg:dash:schema:mpd:2011"
xsi:schemaLocation="urn:mpeg:dash:schema:mpd:2011 http://standards.iso.org/ittf/PubliclyAvailableStandards/MPEG-DASH_schema_files/DASH-MPD.xsd"
type="static"
mediaPresentationDuration="PT14M48S"
maxSegmentDuration="PT1M"
minBufferTime="PT10S"
profiles="urn:mpeg:dash:profile:isoff-live:2011">
<Period
id="1"
duration="PT14M48S">
<BaseURL>dash/</BaseURL>
<AdaptationSet
id="1"
group="1"
contentType="audio"
segmentAlignment="true"
audioSamplingRate="48000"
mimeType="audio/mp4"
codecs="mp4a.40.2"
startWithSAP="1">
<AudioChannelConfiguration
schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011"
value="2" />
<Role schemeIdUri="urn:mpeg:dash:role:2011" value="main" />
<SegmentTemplate
timescale="48000"
initialization="3144-kZT4LWMQw6Rh7Kpd-$RepresentationID$.dash"
media="3144-kZT4LWMQw6Rh7Kpd-$RepresentationID$-$Time$.dash">
<SegmentTimeline>
<S t="0" d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="96256" r="2" />
<S d="95232" />
<S d="3584" />
</SegmentTimeline>
</SegmentTemplate>
<Representation
id="audio=128001"
bandwidth="128001">
</Representation>
</AdaptationSet>
<AdaptationSet
id="2"
group="3"
contentType="text"
lang="en"
mimeType="application/mp4"
codecs="stpp"
startWithSAP="1">
<Role schemeIdUri="urn:mpeg:dash:role:2011" value="subtitle" />
<SegmentTemplate
timescale="1000"
initialization="3144-kZT4LWMQw6Rh7Kpd-$RepresentationID$.dash"
media="3144-kZT4LWMQw6Rh7Kpd-$RepresentationID$-$Time$.dash">
<SegmentTimeline>
<S t="0" d="60000" r="9" />
<S d="24000" />
</SegmentTimeline>
</SegmentTemplate>
<Representation
id="textstream_eng=1000"
bandwidth="1000">
</Representation>
</AdaptationSet>
<AdaptationSet
id="3"
group="2"
contentType="video"
par="960:409"
minBandwidth="100000"
maxBandwidth="4482000"
maxWidth="1689"
maxHeight="720"
segmentAlignment="true"
mimeType="video/mp4"
codecs="avc1.4D401F"
startWithSAP="1">
<Role schemeIdUri="urn:mpeg:dash:role:2011" value="main" />
<SegmentTemplate
timescale="12288"
initialization="3144-kZT4LWMQw6Rh7Kpd-$RepresentationID$.dash"
media="3144-kZT4LWMQw6Rh7Kpd-$RepresentationID$-$Time$.dash">
<SegmentTimeline>
<S t="0" d="24576" r="443" />
</SegmentTimeline>
</SegmentTemplate>
<Representation
id="video=100000"
bandwidth="100000"
width="336"
height="144"
sar="2880:2863"
scanType="progressive">
</Representation>
<Representation
id="video=326000"
bandwidth="326000"
width="562"
height="240"
sar="115200:114929"
scanType="progressive">
</Representation>
<Representation
id="video=698000"
bandwidth="698000"
width="844"
height="360"
sar="86400:86299"
scanType="progressive">
</Representation>
<Representation
id="video=1493000"
bandwidth="1493000"
width="1126"
height="480"
sar="230400:230267"
scanType="progressive">
</Representation>
<Representation
id="video=4482000"
bandwidth="4482000"
width="1688"
height="720"
sar="86400:86299"
scanType="progressive">
</Representation>
</AdaptationSet>
</Period>
</MPD>

28
test/testdata/mpd/unfragmented.mpd vendored Normal file
View File

@ -0,0 +1,28 @@
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<MPD mediaPresentationDuration="PT54.915S" minBufferTime="PT1.500S" profiles="urn:mpeg:dash:profile:isoff-on-demand:2011" type="static" xmlns="urn:mpeg:dash:schema:mpd:2011">
<Period duration="PT54.915S">
<AdaptationSet segmentAlignment="true" subsegmentAlignment="true" subsegmentStartsWithSAP="1">
<Representation bandwidth="804261" codecs="avc1.4d401e" frameRate="30" height="360" id="VIDEO-1" mimeType="video/mp4" startWithSAP="1" width="360">
<BaseURL>DASH_360</BaseURL>
<SegmentBase indexRange="915-1114" indexRangeExact="true">
<Initialization range="0-914"/>
</SegmentBase>
</Representation>
<Representation bandwidth="608000" codecs="avc1.4d401e" frameRate="30" height="240" id="VIDEO-2" mimeType="video/mp4" startWithSAP="1" width="240">
<BaseURL>DASH_240</BaseURL>
<SegmentBase indexRange="913-1112" indexRangeExact="true">
<Initialization range="0-912"/>
</SegmentBase>
</Representation>
</AdaptationSet>
<AdaptationSet>
<Representation audioSamplingRate="48000" bandwidth="129870" codecs="mp4a.40.2" id="AUDIO-1" mimeType="audio/mp4" startWithSAP="1">
<AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
<BaseURL>audio</BaseURL>
<SegmentBase indexRange="832-1007" indexRangeExact="true">
<Initialization range="0-831"/>
</SegmentBase>
</Representation>
</AdaptationSet>
</Period>
</MPD>

32
test/testdata/mpd/url_and_range.mpd vendored Normal file
View File

@ -0,0 +1,32 @@
<?xml version="1.0" ?>
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" profiles="urn:mpeg:dash:profile:isoff-live:2011" minBufferTime="PT10.01S" mediaPresentationDuration="PT30.097S" type="static">
<!-- Created with Bento4 mp4-dash.py, VERSION=2.0.0-639 -->
<Period>
<!-- Video -->
<AdaptationSet mimeType="video/mp4" segmentAlignment="true" startWithSAP="1" maxWidth="768" maxHeight="432">
<Representation id="video-avc1" codecs="avc1.4D401E" width="768" height="432" scanType="progressive" frameRate="30000/1001" bandwidth="699597">
<SegmentList timescale="1000" duration="10010">
<Initialization sourceURL="video-frag.mp4" range="36-746"/>
<SegmentURL media="video-frag.mp4" mediaRange="747-876117"/>
<SegmentURL media="video-frag.mp4" mediaRange="876118-1466913"/>
<SegmentURL media="video-frag.mp4" mediaRange="1466914-1953954"/>
<SegmentURL media="video-frag.mp4" mediaRange="1953955-1994652"/>
</SegmentList>
</Representation>
</AdaptationSet>
<!-- Audio -->
<AdaptationSet mimeType="audio/mp4" startWithSAP="1" segmentAlignment="true">
<Representation id="audio-und-mp4a.40.2" codecs="mp4a.40.2" bandwidth="98808" audioSamplingRate="48000">
<AudioChannelConfiguration schemeIdUri="urn:mpeg:mpegB:cicp:ChannelConfiguration" value="2"/>
<SegmentList timescale="1000" duration="10010">
<Initialization sourceURL="audio-frag.mp4" range="32-623"/>
<SegmentURL media="audio-frag.mp4" mediaRange="624-124199"/>
<SegmentURL media="audio-frag.mp4" mediaRange="124200-250303"/>
<SegmentURL media="audio-frag.mp4" mediaRange="250304-374365"/>
<SegmentURL media="audio-frag.mp4" mediaRange="374366-374836"/>
</SegmentList>
</Representation>
</AdaptationSet>
</Period>
</MPD>

218
test/testdata/mpd/urls_only.mpd vendored Normal file
View File

@ -0,0 +1,218 @@
<?xml version="1.0" ?>
<MPD maxSegmentDuration="PT0H0M10.000S" mediaPresentationDuration="PT0H4M1.728S" minBufferTime="PT1.500S" profiles="urn:mpeg:dash:profile:isoff-main:2011" type="static" xmlns="urn:mpeg:dash:schema:mpd:2011">
<Period duration="PT0H4M1.728S">
<AdaptationSet bitstreamSwitching="true" lang="und" maxHeight="1080" maxWidth="1920" par="16:9" segmentAlignment="true">
<ContentComponent contentType="video" id="1"/>
<Representation audioSamplingRate="44100" bandwidth="200000" codecs="avc3.42c01e,mp4a.40.2" frameRate="25" height="144" id="h264_aac_144p_m4s" mimeType="video/mp4" sar="1:1" startWithSAP="1" width="256">
<SegmentList duration="10000" timescale="1000">
<Initialization sourceURL="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/init/432f65a0.mp4"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/0/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/1/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/2/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/3/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/4/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/5/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/6/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/7/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/8/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/9/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/10/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/11/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/12/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/13/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/14/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/15/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/16/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/17/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/18/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/19/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/20/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/21/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/22/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/23/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_144p_m4s/24/432f65a0.m4s"/>
</SegmentList>
</Representation>
<Representation audioSamplingRate="44100" bandwidth="400000" codecs="avc3.42c01e,mp4a.40.2" frameRate="25" height="240" id="h264_aac_240p_m4s" mimeType="video/mp4" sar="160:159" startWithSAP="1" width="424">
<SegmentList duration="10000" timescale="1000">
<Initialization sourceURL="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/init/432f65a0.mp4"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/0/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/1/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/2/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/3/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/4/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/5/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/6/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/7/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/8/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/9/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/10/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/11/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/12/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/13/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/14/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/15/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/16/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/17/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/18/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/19/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/20/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/21/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/22/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/23/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_240p_m4s/24/432f65a0.m4s"/>
</SegmentList>
</Representation>
<Representation audioSamplingRate="44100" bandwidth="800000" codecs="avc3.42c01e,mp4a.40.2" frameRate="25" height="360" id="h264_aac_360p_m4s" mimeType="video/mp4" sar="1:1" startWithSAP="1" width="640">
<SegmentList duration="10000" timescale="1000">
<Initialization sourceURL="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/init/432f65a0.mp4"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/0/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/1/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/2/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/3/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/4/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/5/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/6/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/7/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/8/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/9/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/10/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/11/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/12/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/13/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/14/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/15/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/16/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/17/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/18/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/19/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/20/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/21/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/22/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/23/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_360p_m4s/24/432f65a0.m4s"/>
</SegmentList>
</Representation>
<Representation audioSamplingRate="44100" bandwidth="1200000" codecs="avc3.42c01e,mp4a.40.2" frameRate="25" height="480" id="h264_aac_480p_m4s" mimeType="video/mp4" sar="320:321" startWithSAP="1" width="856">
<SegmentList duration="10000" timescale="1000">
<Initialization sourceURL="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/init/432f65a0.mp4"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/0/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/1/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/2/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/3/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/4/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/5/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/6/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/7/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/8/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/9/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/10/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/11/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/12/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/13/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/14/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/15/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/16/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/17/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/18/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/19/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/20/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/21/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/22/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/23/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_480p_m4s/24/432f65a0.m4s"/>
</SegmentList>
</Representation>
<Representation audioSamplingRate="44100" bandwidth="1600000" codecs="avc3.42c01e,mp4a.40.2" frameRate="25" height="576" id="h264_aac_576p_m4s" mimeType="video/mp4" sar="1:1" startWithSAP="1" width="1024">
<SegmentList duration="10000" timescale="1000">
<Initialization sourceURL="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/init/432f65a0.mp4"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/0/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/1/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/2/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/3/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/4/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/5/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/6/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/7/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/8/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/9/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/10/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/11/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/12/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/13/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/14/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/15/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/16/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/17/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/18/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/19/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/20/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/21/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/22/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/23/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_576p_m4s/24/432f65a0.m4s"/>
</SegmentList>
</Representation>
<Representation audioSamplingRate="44100" bandwidth="2400000" codecs="avc3.42c01e,mp4a.40.2" frameRate="25" height="720" id="h264_aac_720p_m4s" mimeType="video/mp4" sar="1:1" startWithSAP="1" width="1280">
<SegmentList duration="10000" timescale="1000">
<Initialization sourceURL="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/init/432f65a0.mp4"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/0/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/1/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/2/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/3/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/4/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/5/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/6/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/7/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/8/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/9/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/10/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/11/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/12/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/13/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/14/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/15/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/16/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/17/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/18/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/19/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/20/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/21/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/22/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/23/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_720p_m4s/24/432f65a0.m4s"/>
</SegmentList>
</Representation>
<Representation audioSamplingRate="44100" bandwidth="4400000" codecs="avc3.42c01e,mp4a.40.2" frameRate="25" height="1080" id="h264_aac_1080p_m4s" mimeType="video/mp4" sar="1:1" startWithSAP="1" width="1920">
<SegmentList duration="10000" timescale="1000">
<Initialization sourceURL="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/init/432f65a0.mp4"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/0/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/1/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/2/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/3/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/4/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/5/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/6/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/7/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/8/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/9/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/10/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/11/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/12/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/13/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/14/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/15/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/16/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/17/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/18/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/19/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/20/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/21/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/22/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/23/432f65a0.m4s"/>
<SegmentURL media="../vd_5999c902ea707c67d8e267a9_1503250723/h264_aac_1080p_m4s/24/432f65a0.m4s"/>
</SegmentList>
</Representation>
</AdaptationSet>
</Period>
</MPD>

34
test/testdata/xspf/foo_xspf.xspf vendored Normal file
View File

@ -0,0 +1,34 @@
<?xml version="1.0" encoding="UTF-8"?>
<playlist version="1" xmlns="http://xspf.org/ns/0/">
<date>2018-03-09T18:01:43Z</date>
<trackList>
<track>
<location>cd1/track%201.mp3</location>
<title>Pandemonium</title>
<creator>Foilverb</creator>
<annotation>Visit http://bigbrother404.bandcamp.com</annotation>
<album>Pandemonium EP</album>
<trackNum>1</trackNum>
<duration>202416</duration>
</track>
<track>
<location>../%E3%83%88%E3%83%A9%E3%83%83%E3%82%AF%E3%80%80%EF%BC%92.mp3</location>
<title>Final Cartridge (Nichico Twelve Remix)</title>
<annotation>Visit http://bigbrother404.bandcamp.com</annotation>
<creator>Foilverb</creator>
<album>Pandemonium EP</album>
<trackNum>2</trackNum>
<duration>255857</duration>
</track>
<track>
<location>track3.mp3</location>
<location>https://example.com/track3.mp3</location>
<title>Rebuilding Nightingale</title>
<annotation>Visit http://bigbrother404.bandcamp.com</annotation>
<creator>Foilverb</creator>
<album>Pandemonium EP</album>
<trackNum>3</trackNum>
<duration>287915</duration>
</track>
</trackList>
</playlist>

View File

@ -8,6 +8,7 @@ deps =
passenv = HOME
defaultargs = test --exclude test_download.py --exclude test_age_restriction.py
--exclude test_subtitles.py --exclude test_write_annotations.py
--exclude test_youtube_lists.py
--exclude test_youtube_lists.py --exclude test_iqiyi_sdk_interpreter.py
--exclude test_socks.py
commands = nosetests --verbose {posargs:{[testenv]defaultargs}} # --with-coverage --cover-package=youtube_dl --cover-html
# test.test_download:TestDownload.test_NowVideo

Some files were not shown because too many files have changed in this diff Show More