pepepper/youtube-dl
1
0
Fork 0
mirror of https://github.com/ytdl-org/youtube-dl synced 2025-02-05 10:10:10 +09:00
Code Issues Projects Releases Wiki Activity
18,879 Commits 10 Branches 1,120 Tags 241 MiB
3a4c4ffb50
Commit Graph

682 Commits

Author SHA1 Message Date
jakeogh
1221873dcd add locked_file.flush() 2020-12-19 10:21:58 -07:00
jakeogh
8230a3575e modify sanitize_open() to use locked_file(), preventing silent corruption when a second youtube-dl instance is attempting to write the same file. There is still a corner case, if a .part file is being used (--no-part is not enabled), in that the .part file is closed before it's renamed to remove the .part, in that window, another process could modify the .part file before it's renamed. Using --no-part prevents this corner case. 2020-12-19 10:21:58 -07:00
Josh Soref
71ddc222ad
Fix typos (#27084) 
* spelling: authorization

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: brightcove

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: creation

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: exceeded

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: exception

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: extension

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: extracting

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: extraction

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: frontline

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: improve

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: length

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: listsubtitles

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: multimedia

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: obfuscated

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: partitioning

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: playlist

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: playlists

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: restriction

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: services

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: split

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: srmediathek

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: support

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: thumbnail

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: verification

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: whitespaces

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-11-21 22:00:05 +07:00
Sergey M․
fe07e788bf
[utils] Skip ! prefixed code in js_to_json 2020-11-17 01:30:43 +07:00
Kevin O'Connor
4eda10499e
[utils] Don't attempt to coerce JS strings to numbers in js_to_json (#26851) 
The current logic in `js_to_json` tries to rewrite octal/hex numbers to
decimal. However, when the logic actually happens the `"` or `'` have
already been trimmed off. This causes what were originally strings, that
happen to look like octal/hex numbers, to get rewritten to decimal and
returned as a number rather than a string.

In practive something like:

```js
{
  "0x40": "foo",
  "040": "bar",
}
```

would get rewritten as:

```json
{
  64: "foo",
  32: "bar
}
```

This is problematic since this isn't valid JSON as you cannot have
non-string keys.
 2020-10-18 00:10:41 +07:00
Sergey M․
1d9bf655e6
[utils] Recognize wav mimetype (closes #26463) 2020-09-06 11:19:53 +07:00
Rob
9cd5f54e31
[utils] Fix file permissions in write_json_file (closes #12471) (#25122) 2020-05-20 03:21:52 +07:00
Sergey M․
c380cc28c4
[utils] Improve cookie files support 
+ Add support for UTF-8 in cookie files
* Skip malformed cookie file entries instead of crashing (invalid entry len, invalid expires at)
 2020-05-05 04:21:25 +07:00
Sergey M․
f1a8511f7b
[utils] Add reference to cookie file format 2020-03-10 04:59:02 +07:00
Sergey M․
042b664933
Revert "[utils] Add support for cookies with spaces used instead of tabs" 
According to [1] TABs must be used as separators between fields.
Files produces by some tools with spaces as separators are considered
malformed.

1. https://curl.haxx.se/docs/http-cookies.html

This reverts commit cff99c91d1.
 2020-03-10 04:53:51 +07:00
Sergey M․
cff99c91d1
[utils] Add support for cookies with spaces used instead of tabs 2020-03-08 18:01:32 +07:00
Sergey M․
fca6dba8b8
[YoutubeDL] Force redirect URL to unicode on python 2 2020-02-29 19:08:44 +07:00
Sergey M․
42db58ec73
[utils] Improve str_to_int 2019-12-15 23:15:24 +07:00
Remita Amine
348c6bf1c1 [utils] handle int values passed to str_to_int 2019-11-29 17:39:18 +01:00
Sergey M․
1ced222120
[utils] Add generic caesar cipher and rot47 2019-11-27 02:26:42 +07:00
InfernalUnderling
9d30c2132a [utils] Handle rd-suffixed day parts in unified_strdate (#23199) 2019-11-27 00:08:37 +07:00
Sergey M․
53896ca5be
[utils] Actualize major IPv4 address blocks per country 2019-10-29 06:10:20 +07:00
Sergey M․
824fa51165
[utils] Improve subtitles_filename (closes #22753) 2019-10-18 04:03:53 +07:00
Sergey M․
f7a147e3b6
[utils] Introduce random_user_agent and use as default User-Agent (closes #21546) 2019-06-29 00:32:43 +07:00
Sergey M․
28cc2241e4
[utils] Restrict parse_codecs and add theora as known vcodec (#21381) 2019-06-14 01:56:17 +07:00
Sergey M․
53cd37bac5
[utils] Improve strip_or_none 2019-05-24 00:03:01 +07:00
Sergey M․
3089bc748c
Fix W504 and disable W503 (closes #20863) 2019-05-11 03:57:40 +07:00
Jakub Wilk
fd35d8cdfd [utils] Transliterate "þ" as "th" (#20897) 
Despite visual similarity "þ" is unrelated to "p".
It is normally transliterated as "th":

    $ echo þ-Þ | iconv -t ASCII//TRANSLIT
    th-TH
 2019-05-11 01:42:31 +07:00
Sergey M․
5e1271c56d
[utils] Improve int_or_none and float_or_none (#20403) 2019-03-23 01:08:54 +07:00
Sergey M․
0dc41787af
[utils] Introduce parse_bitrate 2019-03-17 09:07:47 +07:00
Sergey M․
067aa17edf
Start moving to ytdl-org 2019-03-11 04:00:54 +07:00
remitamine
e7e62441cd [utils] strip #HttpOnly_ prefix from cookies files (#20219) 2019-03-03 19:23:59 +07:00
Ales Jirasek
22f5f5c6fc
[malltv] Add extractor (closes #18058) 2019-02-08 00:43:26 +07:00
Sergey M․
fad4ceb534
[utils] Fix urljoin for paths with non-http(s) schemes 2019-01-20 20:22:19 +07:00
Sergey M․
e9a50fba86
[utils] Fix typo 2019-01-07 01:02:34 +07:00
Sergey M․
b7acc83550
[utils] Add language codes replaced in 1989 revision of ISO 639 to ISO639Utils (closes #18765) 2019-01-07 00:57:48 +07:00
Sergey M․
1bab343704
[YoutubeDL] Introduce YoutubeDLCookieJar and clarify the rationale behind session cookies (closes #12929) 2018-12-09 06:47:49 +07:00
Alexander Seiler
aa374bc78e [utils] Fix random_birthday to generate existing dates only 2018-12-02 00:05:15 +07:00
Sergey M․
25d110be30
[utils] Properly recognize AV1 codec (closes #17506) 2018-09-10 02:37:22 +07:00
Sergey M․
9e21e6d96b
[utils] Improve remote address skipping and add support for python 2.6 (closes #17362) 2018-08-29 01:18:03 +07:00
Andrew Udvare
8959018a5f
[utils] Skip remote IP addresses non matching to source address' IP version (closes #13422) 2018-08-29 01:17:53 +07:00
Sergey M․
60c0856223
[utils] Use pure browser header for User-Agent (closes #17236) 2018-08-14 23:27:12 +07:00
Huyuumi
38e87f6c2a [utils] Remove return from __init__ 2018-07-29 05:52:42 +07:00
Sergey M․
af03000ad5
[utils] Introduce url_or_none 2018-07-21 18:03:58 +07:00
Sergey M․
e9c671d5e8
[utils] Allow JSONP with empty func name (closes #17028) 2018-07-21 12:30:18 +07:00
Sergey M․
0685d9727b
[utils] Share JSON-LD regex 2018-07-09 23:43:05 +07:00
Enes
85750f8972 [openload] Improve ext extraction 2018-06-02 00:16:22 +07:00
Remita Amine
261f47306c [utils] fix style id extraction for namespaced id attribute(closes #16551) 2018-05-26 14:38:24 +01:00
Remita Amine
5a16c9d9d3 [utils] keep the original TV_PARENTAL_GUIDELINES dict 2018-05-25 23:12:50 +01:00
Remita Amine
b836118724 [utils] Relax TV Parental Guidelines matching 2018-05-23 12:12:20 +01:00
Sergey M․
5f95927a62
Improve geo bypass mechanism 
* Introduce geo bypass context
* Add ability to bypass based on IP blocks in CIDR notation
* Introduce --geo-bypass-ip-block
 2018-05-02 07:20:59 +07:00
Sergey M․
6cc622327f
[utils] Introduce merge_dicts 2018-04-28 02:47:17 +07:00
Sergey M․
1cc47c6674
[utils] Fix match_str for boolean meta fields 2018-04-24 23:54:49 +07:00
Philipp Hagemeister
f226880c6d [tennistv] Add support for tennistv.com 2018-03-14 09:55:21 +01:00
Sergey M․
b871d7e954
[utils] Add parse_resolution 2018-03-02 23:39:04 +07:00
Sergey M․
befa4708fd
[utils] Fixup some common URL's typos in sanitize_url (closes #15649) 2018-02-19 22:50:23 +07:00
Remita Amine
b12cf31bb1 [cbc] add new extractor for olympics.cbc.ca(closes #15535) 2018-02-19 09:02:23 +01:00
Sergey M․
65220c3bd6
Add support for IronPython 2018-01-28 05:48:18 +07:00
Mike Fährmann
c384d537f8 [util] Improve scientific notation handling in js_to_json (closes #14789) 2018-01-20 22:54:21 +07:00
Philipp Hagemeister
d7cd9a9e84
[utils] Fix youtube-dl under PyPy3 on Windows 2018-01-01 22:48:27 +07:00
Sergey M․
b555ae9bf1
[utils] Add another date format pattern (#14999) 2017-12-16 21:56:16 +07:00
Remita Amine
d4f05d4731 [utils] add sami mimetype to mimetype2ext 2017-12-03 00:04:43 +01:00
Remita Amine
ffe6979ef9 [utils] add hvc1 codec code to parse_codecs 2017-11-27 16:55:27 +01:00
Sergey M․
056653bbb1
[utils] Add support for zero years and months in parse_duration 2017-10-29 07:04:48 +07:00
Sergey M․
6be08ce602
[utils] Use in OnDemandPagedList by default 
Not using cache results in redundant network I/O due to downloading the same pages while using --playlist-items n-m
 2017-10-06 23:13:53 +07:00
Sergey M․
4c54b89e03
Hide experimental phantomjs wrapper 2017-09-24 00:08:27 +07:00
Yen Chi Hsuan
4ed2d7b7d1 Fix flake8 issues after #14225 2017-09-17 13:53:04 +08:00
Yen Chi Hsuan
3869028ffb [utils] Use bytes-like objects in dfxp2srt 
This fixes handling of non-UTF8 TTML subtitles

Closes #14191
 2017-09-16 12:18:38 +08:00
Yen Chi Hsuan
a88d461dff Merge pull request #14225 from Tithen-Firion/openload-phantomjs-method 
Openload phantomjs method
 2017-09-16 02:28:28 +08:00
Sergey M․
c7e327c4d4
[utils] Introduce bool_or_none 2017-09-10 19:08:39 +07:00
Yen Chi Hsuan
95f3f7c20a
[utils] Fix unescapeHTML for misformed string like "&a&quot;" (#13935) 2017-08-19 21:40:53 +08:00
Sergey M․
5b232f46dc
[utils] Skip missing params in cli_bool_option (closes #13865) 2017-08-09 22:28:19 +07:00
Tithen-Firion
feee8d32e4 [phantomjs] add exe version to debug info 2017-08-03 14:17:25 +02:00
Yen Chi Hsuan
609ff8ca19 [utils] Support attributes with no values in get_elements_by_attribute() 2017-07-05 23:27:12 +08:00
Sergey M․
aefce8e6dc
[utils] Use compat_shlex_quote in shell_quote 2017-06-17 23:48:58 +07:00
Sergey M․
b4a3d461e4
[utils] Handle HTMLParseError in extract_attributes (closes #13349) 2017-06-12 01:52:24 +07:00
Sergey M․
2ae2ffda5e
[utils] Improve unified_timestamp 2017-06-11 21:27:22 +07:00
Yen Chi Hsuan
b407d8533d
[utils] Drop an compatibility wrapper for Python < 2.6 
addinfourl.getcode is added since Python 2.6a1. As youtube-dl now
requires 2.6+, this is no longer necessary.

See 9b0d46db11
 2017-05-27 23:05:02 +08:00
Yen Chi Hsuan
5552c9eb0f
[utils] Recognize more patterns in strip_jsonp() 
Used in Youku Show pages
 2017-05-26 21:58:18 +08:00
Sergey M․
60f5c9fb19
[utils] Recognize more audio codecs (#13081) 2017-05-14 12:33:33 +07:00
Yen Chi Hsuan
10c87c151b
[utils] Rename try_multipart_encode to _multipart_encode_impl 
To state that this is an internal function and people should be careful
when using it outside youtube-dl.
 2017-05-06 19:06:18 +08:00
Yen Chi Hsuan
b2ad479d17
[utils] Fix multipart_encode for Python < 3.5 2017-05-05 20:51:59 +08:00
Tithen-Firion
c89267d31a Merge branch 'master' into openload-phantomjs-method 2017-05-04 11:00:06 +02:00
Yen Chi Hsuan
0c26548601
[cda] Implement birthday verification (closes #12789) 2017-05-04 16:26:17 +08:00
Sergey M․
deef31955b
[utils] Improve unified_timestamp 
Seen at http://zaq1.pl/video/xev0e
 2017-04-30 21:45:53 +07:00
Remita Amine
98ce1a3fd3 [utils] add video/mp2t to mimetype2ext 2017-04-30 09:03:10 +01:00
Tithen-Firion
98f9d87381 [phantomjs] Add required version checking 2017-04-29 12:41:42 +02:00
Tithen-Firion
edd9221cd2 [utils] Fix inconsistent output of clean_html 
`\s` in Python 2.x doesn't match unicode whitespace characters by
default
 2017-04-28 17:34:27 +02:00
Tithen-Firion
40e41780f1 [phantomjs] add cookie support 2017-04-25 15:12:54 +02:00
Tithen-Firion
da57ebaf84 [openload] separate PhantomJS code from extractor 2017-04-25 01:06:14 +02:00
Remita Amine
5b995f713b [utils] add support for ttml styles 2017-04-19 14:38:40 +01:00
Sergey M․
a32a9a7ef5
[extractor/common] Add support multiple getters in try_get 2017-04-18 22:39:58 +07:00
Sergey M․
51098426b8
[utils] Introduce expand_path 2017-03-26 02:30:10 +07:00
Sergey M․
4b5de77bdb
[utils] Process bytestrings in urljoin (closes #12369) 2017-03-06 03:57:46 +07:00
Sergey M․
158af5242e
[utils] Carry long doc string 2017-03-01 23:04:02 +07:00
Yen Chi Hsuan
0a5445ddbe [utils] Add bytes_to_long() and long_to_bytes() 
Used in daisuki.net (#4738)

Both are adapted from public domain PyCrypto:
https://github.com/dlitz/pycrypto/blob/master/lib/Crypto/Util/number.py
 2017-02-28 22:10:31 +08:00
Yen Chi Hsuan
f48409c7ac [utils] Add pkcs1pad 
Used in daisuki.net (#4738)
 2017-02-28 22:10:31 +08:00
Sergey M․
4248dad92b Improve geo bypass mechanism 
* Rename options to preffixly match with --geo-verification-proxy
* Introduce _GEO_COUNTRIES for extractors
* Implement faking IP right away for sites with known geo restriction
 2017-02-19 05:10:08 +08:00
Sergey M․
18a0defab0 [utils] Make random_ipv4 return unicode string 2017-02-19 05:10:08 +08:00
Sergey M․
773f291dcb Add experimental geo restriction bypass mechanism 
Based on faking X-Forwarded-For HTTP header
 2017-02-19 05:10:08 +08:00
Pierre Mdawar
bf5b9d859a [utils] Introduce YoutubeDLError base class for all youtube-dl exceptions 2017-02-19 05:10:08 +08:00
Sergey M․
db13c16ef8
[utils] Add support for quoted string literals in --match-filter (closes #8050, closes #12142, closes #12144) 2017-02-15 23:12:10 +07:00
Ricardo Constantino
2f49bcd690
utils.py: Workaround TypeError with Python 2.7.13 in Windows 
Fixes #11540

Tested with Windows Python 2.7.12 and 2.7.13.
 2017-02-11 14:51:28 +00:00
Thomas Christlieb
2af12ad9d2 Introduce get_elements_by_class and get_elements_by_attribute utility functions 2017-02-11 17:16:54 +08:00
Sergey M․
4195096ea8
[utils] Improve comments processing in js_to_json (closes #11947) 2017-02-03 03:04:33 +07:00