edshot99/eBooru
1
0
Fork 0
forked from e621ng/e621ng
Code Issues 11 Releases 1 Activity
11,473 Commits 3 Branches 60 Tags 67 MiB
master2
Commit Graph

1329 Commits

Author SHA1 Message Date
Earlopain
1c6b2dcba9
[Users] Remove special casing for name change feedback 
Part of old danbooru code, these feedbacks where never created on e6.
Also remove name change approve/reject controller actions, it's all automatic
 2022-01-22 16:20:49 +01:00
Earlopain
22d04f6cb1
[Posts] Add postevent tests 2022-01-07 12:29:11 +01:00
Earlopain
bcccec6778
[Replacements] Add test for upload limit changes 2021-06-27 00:21:01 +02:00
Zwagoth
b2b5938d5c
Merge pull request #325 from zwagoth/replacement-search 
[Search] Add pending replacements field
 2021-11-17 00:44:19 -05:00
Earlopain
061176e095
[Setup] Ignore limits when seeding 
This makes the environment config understand booleans. If something
ends with "?" it tries to interpret it as a boolean. A bunch of
existing configs where changed to reflect this change.
 2021-11-14 22:16:36 +01:00
Earlopain
572b61c85c
[Forum] Prevent voting on your own requests 
Removing the votes is still possible but adding new ones  isn't.
Closes #341
 2021-11-02 16:36:05 +01:00
Earlopain
147ec019e8
[Search] Add pending replacements field 2021-10-05 21:14:00 +02:00
Earlopain
1c0c3c2475
[Flags] Allow viewing flags on own uploaded posts 
Apparently  this was a attempt to hide some undesirable information from mods
I personally think this isn't needed and if someone really wanted to
they could just use the other search options which didn't have this limitation
 2021-08-06 19:54:18 +02:00
Earlopain
4b7242005a
[Replacements] Fix broken state when rescaling notes 
After the note will be saved it reloaded the post, throwing the
previous changes like new md5, size etc. away
 2021-06-24 16:22:04 +02:00
Earlopain
eb1f432fc5
Merge branch 'master' into style-changes 2021-06-02 14:28:22 +02:00
Earlopain
8d8a0802a5
[User] Remove memoization of throttles 
This interfered with tests and doesn't seem to give much of an advantage
 2021-05-15 12:36:29 +02:00
Earlopain
9df05e7ba8
[Tests] Add test for forum post throttles 2021-05-14 17:54:46 +02:00
Earlopain
11f4b5e15b
[Tests] Fix user tests 2021-05-14 17:12:51 +02:00
Earlopain
015c238dca
[Config] Remove member_comment_time_threshold 
This was only stubbed in tests and not used anywhere else
 2021-05-14 17:07:43 +02:00
Earlopain
a023c2806e
[Tests] Fix forum post/topic tests 2021-05-14 15:04:03 +02:00
Earlopain
bb56d14221
[Favorites] Use limit parameter and update tests 
Some tests still fail because of is_last_page/is_first_page, but the syntax is fine now

Fixes #267
 2021-05-09 18:57:01 +02:00
Zwagoth
af7f7f9d10
Merge pull request #256 from Earlopain/remove-useless-config 
Remove useless config entries
 2021-04-18 17:41:12 -04:00
Kira
96c7a76bb8 [Replacements] Do not post comment on replacement 2021-04-16 18:05:22 -07:00
Kira
3bcaa6dacf [Replacement] Add post replacement promote test 2021-04-07 10:33:19 -07:00
Kira
173294fb56 [Replacements] Testing 2021-04-07 10:33:19 -07:00
Kira
2298beaa6c [Tests] Fix post replacement controller tests 2021-04-07 10:33:19 -07:00
Kira
c2e6b16796 Tests and fixes 2021-04-07 10:33:18 -07:00
Earlopain
e7b03a84e9
[Config] remove dead keys 2021-04-03 20:14:15 +02:00
Kira
8ea9419045 [Pools] Fix pool name validator for '_-_' 
A bad regex was added which was flagging pool names as invalid
if they contained _-_ in them or ' - '. As these are heavily used
in pool names they should be allowed. But consequitive characters
should still be not allowed.
 2021-03-11 00:06:20 -08:00
Kira
5c90693338 [Posts] Allow copying source/tags on delete 2021-03-10 22:39:12 -08:00
Kira
9c6939c302 [Tag Aliases] Don't update locked tag categories on alias approval. 2021-02-28 13:30:28 -08:00
Kira
784890ca3e [Posts] Fix tag set presenter using broken memoist gem 2021-02-25 12:08:09 -08:00
Kira
76d1999ade [Rails] Upgrade to rails 6.1 + fix tests 2021-02-25 12:08:09 -08:00
Kira
e685fb87c2 [Tests] Update tests and fix deprecations 2021-02-25 12:08:09 -08:00
Kira
12cdec9642 [Tests] Remove test for removed class 2021-02-25 12:08:09 -08:00
Kira
fd635ff197 [Tests] Fix vote tests 2021-02-25 12:08:09 -08:00
Kira
e2a7fa1e27 [Tests] Fix favorite tests 2021-02-25 12:08:09 -08:00
Kira
2f416b51b4 [Tests] Fix pool tests 
The atomic add! and remove! methods were not properly resetting
the skip_sync property once the pool was updated. This was causing
pool test behavior to fail because later syncronize methods would
fail to sync.
 2021-02-25 12:08:09 -08:00
Kira
dbb5870d93 [Tests] Fix pool related errors in tests 2021-02-25 12:08:09 -08:00
Kira
46fe5f1164 [Pools] Pool tests 2021-02-25 12:08:09 -08:00
Kira
c246cd68a2 More test fixes 2021-02-25 12:08:08 -08:00
Kira
8364af65c5 More test fixing 2021-02-25 12:08:08 -08:00
Kira
58e8fe22fd Tests and fixes 2021-02-25 12:08:08 -08:00
Kira
32e81770ef Update tests 2021-02-25 12:00:09 -08:00
Kira
4d996fed54 Update allowed tag naming patterns 2020-12-20 11:31:52 -08:00
Kira
3d82cc3430 [Tag Relations] Remove secondary validations 2020-07-17 01:34:12 -07:00
Kira
160b0cb22e Remove saved searches 2019-10-21 02:57:03 -07:00
Kira
6f74ef28bf Remove broken bulk revert tool 2019-09-28 22:10:23 -07:00
Kira
6fb1e3ceaf Fix up users being broken in tests and fix a few basic tests 2019-09-09 12:58:18 -07:00
Kira
55cb64acf7 Fix deprecated use of update_attributes 2019-09-09 12:58:18 -07:00
evazion
aae3a8edfb artists: reduce queries in artist summaries. 
Avoid a few queries when searching for single artist tags.
 2019-09-05 06:03:33 -07:00
evazion
ae5d039745 app controller: move safe mode from app controller to SessionLoader. 2019-08-28 21:09:37 -07:00
evazion
4e39ef572e Fix #4125: Detect forum and comment spam. 2019-08-28 20:51:14 -07:00
evazion
30ee20a570 dmails: factor out spam detector service. 2019-08-28 20:48:31 -07:00
evazion
531b5c9845 models: drop various unused #named methods. 2019-08-18 13:35:11 -07:00
Kira
a63a5c3131 Remove caching from tag aliases 2019-08-13 06:04:42 -07:00
evazion
ef806bf340 tests: fix ip ban test regression. 2019-08-13 03:28:27 -07:00
evazion
6beb8a81e1 Fix #4106: Allow moderators to IP ban subnets. 2019-08-13 03:24:26 -07:00
evazion
2cd0355665 Fix #4130: Remove tag alias corrections. 2019-08-13 03:18:39 -07:00
evazion
ec0ae03aef nijie: update for new image urls. 
Nijie moved from this:

    https://pic03.nijie.info/nijie_picture/236014_20170620101426_0.png (page: https://www.nijie.info/view.php?id=218856)

to this:

    https://pic.nijie.net/03/nijie_picture/236014_20170620101426_0.png (page: https://www.nijie.info/view.php?id=218856)
 2019-08-10 05:40:31 -07:00
evazion
3931fb454b Fix #4053: Add disapproval index improvements. 
Add search form to /moderator/post/disapprovals.
 2019-08-10 05:39:04 -07:00
evazion
1e7b70c476 Fix #4107: Can't create artist entry if tag already has a wiki #4107 2019-08-10 05:33:43 -07:00
Kira
4024108f1c Convert builder -> janitor for user rank 2019-06-29 12:58:28 -07:00
Kira
9fa294ba31 Convert gold -> privileged for user level 2019-06-29 08:45:58 -07:00
Zwagoth
4a46874a50 Revert "Remove post disapproval" 
This reverts commit c6df9a33e0.
 2019-04-11 23:39:44 -04:00
Kira
5083e6d92e Remove favorite groups 2019-03-23 00:57:37 -07:00
Kira
f4f030f726 Password migration/upgrade 
NOTE: All existing passwords in development no longer work after
this change! Change your users password using the rails console.

Automatically convert and ugrade old passwords to using bcrypt
Removed the seemingly pointless transformation and hashing on top
of the actual password with a static salt.
Disabled logging in using password hashes, because that's just not
secure in any way, and negates cracking passwords at all.
Disabled sending the password hash to the client as a cookie, even
if it was signed.
Disabled legacy API logins.
 2019-02-21 21:10:20 -08:00
Kira
c6df9a33e0 Remove post disapproval 
This simplifies the approval system to be a single user with an
auditing table. The disapproval system was confusing and outright
post deletion replaces it on e6.

Slightly changes the rules of how post approvals work to remove
a few limits that are not reflected on e6.
 2019-02-19 20:20:15 -08:00
Kira
3e8483eaf0 Tests for at least some of the new jobs 2019-02-18 16:29:13 -08:00
Kira
43cad3f5e7 Remove post banning functionality 
The distinction between file deletion and banning is pretty weird
and there are a bunch of edge cases that make this not worth
trying to maintain a distinction.

Left in the automatic implication code on banned artists because
it looked useful for automating some of the basic tasks that are
done surrounding DNP artists.
 2019-02-14 01:44:54 -08:00
Kira
f9d9fc0c72 Unlink voting and favorite actions 
E621 uses a decoupled voting and favorite system to give users
more choices about how they interact with these systems.
 2019-02-13 20:36:55 -08:00
Kira
cfd65dc20b Add DNS email blacklist support 
Look up the MX records for domains when validating blacklists.

A common trick is to register a domain and point it at a spam
email provider to bypass lists.
 2019-02-09 18:57:35 -08:00
Kira
0bd76687a6 Add email blacklist controller 2019-02-09 18:42:43 -08:00
Kira
143e428bab Add email blacklist to user model 
Thi allows prevent specific email addresses from being used during
signup or email change.
 2019-02-09 14:00:37 -08:00
Kira
21aaa2a185 Add upload whitelist model + controller 
Allow preventing uploads from unknown sources by having a whitelist
of allowed domains and patterns that can be directly fetched from.
 2019-02-06 21:08:03 -08:00
Kira
4d1f436266 Add locked tags feature 
It is desirable for administrators to be able to shape what tags
can and can not be applied to specific posts to cut down on how
many tickets need to be handled about tag vandalism.

TODO: Tell users which tags were not updated if they hit any
locked tags.
 2019-02-02 18:03:47 -08:00
Albert Yi
1550538dc1 Tag change notices 
This adds a small notice at the bottom of post searches if a single tag search is the target of any tag change request.
 2019-01-22 17:22:07 -08:00
evazion
8cadef2dd7 pixiv: fix illust id parsing (fix #4043). 
* Tighten up illust id parsing to avoid misparsing ids from
  non-illust urls (sketch urls and novel urls).

* Move id parsing tests from post_test.rb to sources/pixiv_test.rb.

* Drop support for touch.pixiv.net urls. These urls are no longer used
  by Pixiv and aren't present as the source of any posts on Danbooru.
 2019-01-13 14:28:51 -06:00
Albert Yi
f33b23d035 add post count estimates for bulk update requests 2019-01-09 15:54:55 -08:00
Albert Yi
844b1a81ba
Merge pull request #4039 from evazion/fix-4038 
Fix #4038: Attempting to create an IP-ban bans the creator.
 2019-01-09 14:47:32 -08:00
Albert Yi
d97622d1bb
Merge pull request #4007 from evazion/fix-4004 
Fix #4004: Add additional order by metatags for posts
 2019-01-09 14:43:15 -08:00
evazion
80f43f9a7c Fix #4038: Attempting to create an IP-ban bans the creator. 
* Rename comments.ip_addr to comments.creator_ip_addr.
* Fix belongs_to_creator to not clobber ip_addr field.
 2019-01-09 16:20:34 -06:00
Albert Yi
8b0af19f7f fixes #3824: render tag requests dynamically in the forum post 
refactoring
 2019-01-08 12:57:37 -08:00
evazion
5c54c61d65 Fix #4035: The Related Tag JSON endpoint is returning the wrong information 
* Fix /related_tags.json to return a list of tags in the `other_wikis` field.

* Add support for /related_tags.xml.
 2019-01-07 13:30:10 -06:00
evazion
04d5b16da7 pixiv: fix failure to upload bad pixiv id images (fix #4031) 
Bug: Uploading bad pixiv id images failed because the pixiv strategy
raised a BadIDError exception when the upload service checked for the
ugoira frame data.
 2019-01-03 18:01:20 -06:00
evazion
886096b47a BURs: fix remove alias command removing inactive aliases. 
Fix the `remove alias` and `remove implication` commands to only remove
active aliases or implications, not pending/deleted/retired ones.
 2019-01-03 16:30:39 -06:00
Albert Yi
f78ff57884 fix forum topic test 2019-01-02 12:33:56 -08:00
Albert Yi
6f30c99ac3 fix tag alias correction tests 
also removes references to distributed memcached servers in tag alias correction
 2019-01-02 11:52:35 -08:00
evazion
d393d67a9b aliases/implications: don't destroy removed aliases (#4024). 2018-12-31 17:07:14 -06:00
evazion
f5116c5ce2 aliases/implications: allow duplicate inactive aliases/implications. 
Allow multiple pending, deleted or retired aliases/implications for the
same tag.

This is so that deleted or retired aliases can be resubmitted as new
pending requests.
 2018-12-31 17:07:14 -06:00
evazion
44a54f75cd tests: don't automatically approve aliases/implications on creation. 
Automagically approving inactive aliases makes testing more difficult.
 2018-12-31 17:07:14 -06:00
evazion
341403c7bb tests: fix broken bulk update request test. 2018-12-31 17:07:14 -06:00
evazion
6a77b68b74 tumblr: fix tests. 2018-12-27 15:03:11 -06:00
evazion
0f513d1a1b twitter: include intent url in new artist entries (#4028). 2018-12-27 15:03:11 -06:00
evazion
2129e60b2b pixiv: include stacc url in new artist entries (#4028). 2018-12-27 15:03:11 -06:00
evazion
2170961f47 artists: improve prefilling of new artist form (#4028) 
* When creating an artist by clicking the '?' next to the artist tag in
  the tag list, prefill the new artist form by finding the artist's last
  upload and fetching its source data.

  Previously we filled the urls with the source of the artist's last
  upload, which was wrong because it was usually a direct image URL (#3078).

* Fix the other names field not escaping spaces within names to underscores.

* Fix the other names field being potentially prefilled with duplicate names.
 2018-12-27 15:03:11 -06:00
evazion
286bf2f285 artists: filter out duplicates from other names (#4028). 2018-12-27 15:03:11 -06:00
evazion
3c9b26c247 tests: fix *_(cosplay) tagging test (fixup for 51b08d224). 2018-12-26 20:16:59 -06:00
evazion
719f556dd5 aliases: warn about conflicting wikis when alias is requested. 
* Warn about conflicting wiki pages during secondary validation.

* Only warn about missing wiki if both tags are missing a wiki page.
 2018-12-26 17:30:07 -06:00
evazion
365e04b5f9 implications: fix validation of circular implications. 
Fix bug where circular chains like `a -> b && b -> c && c -> a` were
allowed.
 2018-12-26 17:30:07 -06:00
evazion
51b08d2243 implications: ensure antecedent/consequent tags exist. 2018-12-26 17:30:07 -06:00
evazion
c27ba02b88 aliases/implications: clean up validations. 
* Don't return true/false in validations (does nothing).
* Prefer `errors[:base]` over `self.errors[:base]`.
* Add antecedent_wiki / consequent_wiki associations.
* Factor out antecedent_and_consequent_are_different validation.
 2018-12-26 17:30:07 -06:00
evazion
3e382bd831 Fix #4014: Incorrect tag type counts on posts. 2018-12-19 12:20:29 -06:00
evazion
c700ea4b5f Fix #4016: Translated tags failing to find some tags. 
* Normalize spaces to underscores when saving other names. Preserve case
  since case can be significant.

* Fix WikiPage#other_names_include to search case-insensitively (note:
  this prevents using the index).

* Fix sources to return the raw tags in `#tags` and the normalized tags
  in `#normalized_tags`. The normalized tags are the tags that will be
  matched against other names.
 2018-12-16 11:37:57 -06:00
Albert Yi
4706cae114 Add user permissions for flagging and for giving user feedback 2018-12-11 17:08:31 -08:00
evazion
ea9c3576d8 search: add synonyms for *_count metatags. 
Allow e.g. `deleted_comments` as a synonym for `deleted_comment_count`.
 2018-12-11 18:10:20 -06:00
evazion
4a1f0523a4 search: add comment_count, note_count metatags (#4004). 
Add these metatags:

* comment_count
* deleted_comment_count
* active_comment_count
* note_count
* deleted_note_count
* active_note_count
* order:comment_count
* order:deleted_comment_count
* order:active_comment_count
* order:note_count
* order:deleted_note_count
* order:active_note_count
 2018-12-11 18:10:20 -06:00
Albert Yi
79dfa6c28f fix migration for circleci 2018-12-10 17:51:57 -08:00
evazion
7ae27c370d Fix #3995: Automatically imply *_school_uniform -> school_uniform. 2018-12-05 12:19:27 -06:00
evazion
fd2d56a388 Fix #3996: Some direct artstation urls cause the bookmarklet to fail. 
* Don't fail on urls that don't contain the project id (direct image urls).
* Don't fail when the work is deleted.
* Parse artist name from url when possible. This way the artist finder works on bad_artstation_id posts.
* Set canonical source url to `https://artist.artstation.com/projects/12345` instead of
  `https://www.artstation.com/artwork/1235` (this way we preserve the artist name).
* Cache api call.
* Include api call results in /source.json.
 2018-12-03 15:48:29 -06:00
evazion
e8c3d92696 Fix #3977: DeviantArt: Support new wixmp.com urls. 2018-11-30 00:10:59 -06:00
evazion
8da3474a8b Fix #3994: Some deviantart images mistakenly choosing preview image. 2018-11-29 23:01:08 -06:00
Albert Yi
60cee5f452
Merge pull request #3989 from evazion/fix-3987 
Wiki pages: convert other_names column to array (#3987)
 2018-11-19 16:23:32 -08:00
evazion
d82418ed43 Fix #3988: ordpool:<name> fails for large pools. 
Also fixes ordpool:<name> not returning all posts in the correct order
when searching for series pools that contain duplicate posts.
 2018-11-16 22:48:17 -06:00
evazion
41ff05c121 artists: convert other_names to array (#3987). 2018-11-15 14:31:16 -06:00
Albert Yi
0508b127fd continue refactoring savedsearch 2018-11-15 12:06:13 -08:00
evazion
741462ae68 artist versions: convert other_names, url_string to arrays (#3987). 2018-11-14 14:25:02 -06:00
evazion
fe2698a011 tag implications: convert descendant_names to array (#3987). 2018-11-14 12:51:18 -06:00
evazion
308a5021b4 wiki pages: convert other_names to array (#3987). 2018-11-13 19:18:11 -06:00
evazion
bea8c2a4b8 nijie: fix failure to handle certain image urls. 
Fix IMAGE_URL regex not matching urls of this form:

* https://pic04.nijie.info/nijie_picture/diff/main/287736_161475_20181112032855_1.png

This caused the illust id to not be parsed from the url, which led to `#image_url`
returning nil, which led to uploads failing because the url to download was missing.
 2018-11-12 18:04:07 -06:00
evazion
8f6c710c6b tests: fix translated tags test failures. 2018-11-12 18:04:07 -06:00
evazion
7524d52276 Fix #3985: Uploads: 405 Method Not Allowed. 2018-11-11 20:18:21 -06:00
evazion
6a7cd6ce8e Fix #3984: Twitter: undefined method `first' for nil:NilClass. 
Fix Sources::Strategies::Twitter#image_urls to return an empty array
instead of nil when the tweet doesn't contain any images.
 2018-11-11 17:41:32 -06:00
evazion
eade33fa7c Fix #3981: Inconsistency between posts and pools. 2018-11-10 12:37:08 -06:00
evazion
115ed16a96 pools: store post_ids as array instead of string (fix #3979) 2018-11-08 15:09:31 -06:00
evazion
1281481548 Fix #3978: Pool name/category validations not being enforced. 2018-11-07 16:10:16 -06:00
evazion
29b6121a07 pools: refactor #neighbors + fix broken #neighbors tests. 2018-11-04 19:40:57 -06:00
evazion
5cf6a43918 sources: fix sources sometimes choosing wrong strategy (fix #3968) 
Fix sources choosing the wrong strategy when the referer belongs to a
different site (for example, when uploading a twitter post with a pixiv
referer).

* Fix `match?` to only consider the main url, not the referer.

* Change `match?` to match against a list of domains given by the `domains` method.

* Change `match?` to an instance method.
 2018-11-04 13:00:17 -06:00
evazion
f5012464ab Fix #3965: Extraneous API attributes. 
Remove the updater_id/updater_ip_addr virtual attributes from
pools/notes. Juss pass them in as params to create_version instead.
 2018-10-30 15:41:17 -05:00
Albert Yi
5a13d06501 rescue failed network calls on artist test 2018-10-25 13:47:16 -07:00
Albert Yi
65a3e53667 eliminate extraneous assertion 2018-10-25 11:03:59 -07:00
Albert Yi
7d0b32d489 fix reltagquery test 2018-10-25 11:02:07 -07:00
Albert Yi
120a3d6402 fix post test 2018-10-24 17:35:01 -07:00
Albert Yi
9899199f33 eliminate weeklymaintenance file 2018-10-23 17:50:28 -07:00
Albert Yi
2aabaf56d8
Merge pull request #3951 from evazion/fix-maintenance-logging 
Improve daily maintenance error handling
 2018-10-15 14:20:44 -07:00
evazion
e329764276 Fix #3952: Adding animated_gif tag to a ugoira post breaks the player. 2018-10-10 23:50:36 -05:00
evazion
5201954413 maintenance: consolidate maintenance tasks in logical/maintenance.rb. 2018-10-09 20:11:23 -05:00
evazion
53a51310a3 tumblr: add canonical url tests (#3385). 2018-10-09 12:55:48 -05:00
evazion
16b1b72da5 tumblr: fix video urls not being recognized. 2018-10-09 12:44:59 -05:00
evazion
184a5ebf3e tumblr: fix _640 images not being recognized (#3944). 
Fixes _640 images not being matched by the IMAGE regex and therefore not
being rewritten to the largest size.
 2018-10-09 12:44:59 -05:00
evazion
d874c68419 tumblr: fix image_urls when api data is unavailable. 2018-10-09 12:44:59 -05:00
evazion
b0d7d90103 tumblr: extract info from url when api data is unavailable. 
Derive the artist name / profile url / page url from the source URLs when
the API response is unavailable because the Tumblr post was deleted.

This fixes the artist finder to work on bad_tumblr_id posts.
 2018-10-09 12:44:59 -05:00
evazion
0c31a5d6a9 tumblr: don't fail when api data is unavailable (#3948). 
The api data is unavailable when the work is deleted (bad_tumblr_id), or
when the source is a direct image url with no page referer.
 2018-10-09 12:44:59 -05:00
evazion
4c55c809b0 tumblr: don't fail when api key isn't configured. 2018-10-09 12:44:59 -05:00
evazion
c8d538f618 moebooru: delegate to substrategy based on post source (#3911). 
If the yande.re or konachan.com post has a source from a supported site,
for example Pixiv or Twitter, then delegate the artist and commentary
lookup to that substrategy.

Only do this for sources from recognized sites, not the null strategy.
 2018-10-06 14:27:49 -05:00
evazion
e5a4193dd4 moebooru: support batch bookmarklet previews (#3911). 2018-10-06 00:58:22 -05:00
evazion
fdb6e4ecee moebooru: rewrite konachan urls for Post#normalized_source (#3911). 2018-10-06 00:58:22 -05:00
evazion
864349dc7b moebooru: fetch tags (#3911). 2018-10-06 00:58:22 -05:00
evazion
9301bf2f11 tests: fix post expungement test. 
Delay jobs so the uploaded file isn't deleted immediately (see f6c928e660).
 2018-10-06 00:58:22 -05:00
evazion
bb5f291112 artists: don't create new version when nothing changed. 
Fix an issue where saving an artist entry without changing anything
would create a new artist version.
 2018-10-04 20:01:38 -05:00
evazion
03cc3dfa50 artists: fix editing invalid urls in artist entries (fix #3720, #3927, #3781) 
Convert to an autosave association on urls. This ensures that when we
save the artist we only validate the added urls, not bad urls that we're
trying to remove, and that url validation errors are propagated up to
the artist object.

This also fixes invalid urls being saved in the artist history despite
validation failing (#3720).
 2018-10-04 19:49:16 -05:00
evazion
bd3fb7d70e Post#normalized_source: fix for yande.re urls. 
Fix regex for yande.re urls like this:

    https://files.yande.re/image/b66909b940e8d77accab7c9b25aa4dc3/yande.re%20377828.png
 2018-10-01 20:03:21 -05:00
evazion
0c5452cdc3 Fix #3937: Blank lines in tagbox if certain taggroups are absent 2018-10-01 19:47:39 -05:00
evazion
2ae7ec42df Post#fast_count: raise min cache lifetime to 3 minutes (#3925). 2018-10-01 10:38:15 -05:00
evazion
4425150298 Post#fast_count: fix cache expiry not being set (#3925). 2018-10-01 10:38:15 -05:00
evazion
2cc4e35cc9 Fix #3930: Can't remove children from a parent post through child: metatag. 
Add `child:none` and `-child:123` edit metatags. Allow using ranges with
these metatags (e.g. `-child:1..10`, `child:1,3,5`).
 2018-09-29 20:42:38 -05:00
evazion
984cd0432c Fix #3928: fix case sensitivity in metatags. 2018-09-27 23:42:30 -05:00
evazion
f5c9616f62 tests: fix broken tests. 2018-09-24 17:34:08 -05:00
evazion
d9063a9f2a nijie: support preview urls (#3919). 2018-09-24 17:08:37 -05:00
evazion
5525bbe1ca nijie: normalize all thumbnail urls (#3919). 2018-09-23 20:08:14 -05:00
evazion
d294514dc0 nijie: don't crash on invalid urls or deleted works (#3919). 2018-09-23 20:08:14 -05:00
evazion
b6228505aa nijie: fix page_url method. 
The id in a bare image url is the member id, not the illust id.
 2018-09-23 20:08:13 -05:00
evazion
39f9e01b13 nicoseiga: fix canonical_url to use the image url. 2018-09-22 11:07:18 -05:00
evazion
09a8198979 /artists: add wildcard, regex search to url field (#3900) 
Allow searching the URL field by regex or by wildcard.

If the query looks like `/twitter/` do a regex search, otherwise if it
looks like `http://www.twitter.com/*` do a wildcard search, otherwise if
it looks like an url do an artist finder search, lastly if it looks like
`twitter` do a `*twitter*` search.
 2018-09-21 21:19:01 -05:00
evazion
a4608daf38 /artists: add more search options for other names, group name. 
Add these search params:

* /artists?search[<field>]=
* /artists?search[<field>_eq]=
* /artists?search[<field>_not_eq]=
* /artists?search[<field>_like]=
* /artists?search[<field>_not_like]=
* /artists?search[<field>_ilike]=
* /artists?search[<field>_not_ilike]=
* /artists?search[<field>_regex]=
* /artists?search[<field>_not_regex]=

where `<field>` can be `name`, `group_name`, or `other_names`.

Remove these search params:

* /artists?search[name_matches]=
* /artists?search[other_names_match]=
* /artists?search[group_name_matches]=

`/artists?search[<field>_like]=` effectively does the same thing that
these searches did.
 2018-09-21 20:55:14 -05:00
evazion
f917b83d6f /artists: drop deprecated search syntax, add regex search for names. 
Drop support for the following pseudo-metatags in the Name field in the
artists search form:

* name:<name>
* other:<other name>
* group:<group name>
* status:banned
* status:active
* http://www.example.com

Instead, make the Name field do a wildcard search against the artist
name, group name, or other names. If the query looks like `/regex/`,
then do a regex search against any of these names.

/artists?search[name] now does a literal exact match and
/artists?search{any_name_matches] does the above wildcard/regex search.
 2018-09-21 20:51:53 -05:00
evazion
237ab9b782 dmail filters: fix filters being case sensitive. 2018-09-20 19:51:59 -05:00
evazion
03abbd0683 Fix #2894: Use [[:space:]] instead of \s in regexes. 2018-09-20 19:24:38 -05:00
evazion
29cdaddd86 PostSetPresenters::Post#related_posts: clean up metatag parsing (#2894). 
* Fix `#related_tags` to use `Tag.has_metatag?`.
* Fix Tag::SUBQUERY_METATAGS and Tag::METATAGS to be arrays instead of regexes.
 2018-09-20 19:23:47 -05:00
evazion
6fe883c316 posts index: clean up limit:<n> / order:random parsing (#2894) 
* Move the limit:<n> / order:random metatag parsing from the controller
  to the post set.

* Introduce `Tag.has_metatag?` and use it to parse these metatags
  instead of using a regex (#2894).
 2018-09-20 19:23:47 -05:00
evazion
958a9f505b moebooru: rewrite sample urls + support bookmarklet on html page. 
* Fixes #2942: Add Moebooru Rewrite for Sample Images.
* Addresses #3911: Improve Moebooru support.
 2018-09-19 23:32:21 -05:00
evazion
f4e08ef30d Downloads::File: fix SSRF inside is_cloudflare? (#2498). 
Fixes the banned IP check not being applied when sending the HEAD
request for is_cloudflare?.

Also fixes the `#size` method not using the uncached url (which meant
the bookmarklet could report the wrong filesize on artstation uploads).
 2018-09-19 20:11:53 -05:00
Albert Yi
9e1e73ec4c migrate blacklists during mass updates (#2251) 2018-09-19 16:25:06 -07:00
evazion
2f17082e73 Downloads::File: fix SSRF when following redirects (#2498). 
Fixes the banned IP check not being applied when following redirects:

  http://danbooru.donmai.us/uploads/new?url=http://httpbin.org/redirect-to%3Furl=http://127.0.0.1/test.jpg
 2018-09-18 16:00:03 -05:00
evazion
99221e4028 Downloads::File: fix SSRF attack when fetching remote size (#2498). 
Fixes the banned IP check not being applied when fetching the remote
file size. This allowed one to trick Danbooru into sending HEAD requests
to private IPs:

  http://danbooru.donmai.us/uploads/new?url=http://127.0.0.1/test.jpg
 2018-09-18 12:16:27 -05:00
evazion
9cdfbba6c2 Fix #3910: Corrupted images during upload. 
Use a fresh tempfile for each download attempt instead of reusing the same
file (and having to rewind/truncate it after each failed attempt).
 2018-09-18 10:01:44 -05:00
evazion
d3c135ec72 Downloads::File#http_get_streaming: clean up retry logic. 
Replace handrolled retry logic with retriable gem (already pulled in by another gem).
 2018-09-18 09:44:15 -05:00
evazion
96e89cecfb tests: move twitter canonical url test. 2018-09-17 23:27:53 -05:00
Albert Yi
aea2d77c3a
Merge pull request #3907 from evazion/fix-3900 
#3900: Allow to search for urls associated with artists using wildcards
 2018-09-17 12:03:15 -07:00
evazion
4a99cb098f moebooru: use the image url as the canonical url. 2018-09-16 21:00:11 -05:00
evazion
d9ce953752 Fix #3906: Moebooru strategy raises NotImplementedError. 2018-09-16 21:00:11 -05:00
evazion
cae78fa8ee moebooru: move tests from unit/downloads to unit/sources. 2018-09-16 21:00:11 -05:00
evazion
f135a7c064 twitter: normalize canonical urls. 
Normalize http://mobile.twitter.com to http://twitter.com in canonical urls.
 2018-09-16 15:03:47 -05:00
evazion
bd47641601 twitter: don't fail when api key isn't configured. 2018-09-16 15:03:47 -05:00
evazion
325120ee51 twitter: fix parsing of the artist name from the url. 
Fixes URLs like https://twitter.com/intent/user?user_id=123 being
incorrectly normalized to http://twitter.com/intent/ in artist entries.

Also fixes the artist name to be taken from the url when it can't be
obtained from the api (when the tweet is deleted).
 2018-09-16 15:03:23 -05:00
evazion
c9b3c8d217 artist urls: add tests for controller + artist url search. 2018-09-15 19:58:55 -05:00
evazion
761f2649af artists: add more invalid name tests (#3901). 2018-09-14 13:32:45 -05:00
Albert Yi
f487b2a2c6
Merge pull request #3889 from evazion/fix-replace-artist-finder 
Cleanup artist finder
 2018-09-12 11:44:42 -07:00
evazion
fbd5f6b7f2 pixiv: fix preview_urls for ugoiras (#3891). 2018-09-12 00:43:10 -05:00
evazion
37fc215d75 pixiv: fix preview_urls to use correct url (#3891). 2018-09-11 23:55:46 -05:00
evazion
583f8457f0 artists: clean up artist finding logic. 
Rename Artist#find_all_by_url to url_matches and drop previous
url_matches method, along with find_artists and search_for_profile.

Previously find_artists tried to lookup the url, referer url, and profile
url in turn until an artist match was found. This was wasteful, because
the source strategy already knows which url to lookup (usually the profile
url). If that url doesn't find a match, then the artist doesn't exist.
 2018-09-11 20:14:46 -05:00
Albert Yi
a5df178bcc
Merge pull request #3886 from r888888888/source-api-caching 
cache api clients
 2018-09-11 17:34:25 -07:00
Albert Yi
4972c998f8 rely on preview urls if available for gallery 2018-09-11 15:06:12 -07:00
Albert Yi
266c7c0d5b cache api clients 2018-09-11 14:19:17 -07:00
Albert Yi
f16c3a3f40 fix nijie specs 2018-09-11 13:27:00 -07:00
evazion
9a980367f6 twitter: normalize artist commentaries to nfkc (#3719) 
Fixes hashtags not being interpreted when the author uses a fullwidth
number sign (＃, U+FF03).

ref: https://github.com/r888888888/danbooru/issues/3719#issuecomment-419535610
 2018-09-10 21:45:50 -05:00
evazion
c9300cc54e sta.sh: add tests + docs. 2018-09-10 19:26:53 -05:00
evazion
0d7b7e6e85 twitter tests: fix dead tweet. 2018-09-09 14:10:43 -05:00
evazion
b924c2bb9c nijie: fix artist url normalization. 2018-09-09 13:17:52 -05:00
evazion
a67edb8783 deviantart: fix artist finder for artist names with underscores. 
Fix the artist finder for urls like this:

  https://orig00.deviantart.net/4274/f/2010/230/8/a/pkmn_king_and_queen_by_mikoto_chan.jpg

that don't contain a deviantart id but do contain the artist name.
 2018-09-07 12:23:48 -05:00
evazion
610391205f deviantart: fix artist finder for profile urls missing the 'www'. 
Fix the artist finder to work when the profile url in the artist entry
is missing the 'www'. Example:

  https://deviantart.com/noizave
  https://www.deviantart.com/noizave
 2018-09-07 11:36:48 -05:00
evazion
aee1906761 Fix #3738: Artist URL search should be case-insensitive for domains. 2018-09-05 19:14:24 -05:00
Albert Yi
69b7c9efd0
Merge pull request #3838 from evazion/fix-3837 
Fix #3837: Remove expired bans.
 2018-09-05 12:03:46 -07:00
Albert Yi
a41d006380 fix pool ordering js (#3869), fix forum topic tests 2018-09-05 12:01:59 -07:00
Albert Yi
314341773c fix tests 2018-09-04 18:23:49 -07:00
Albert Yi
a5943de418
Merge pull request #3868 from evazion/fix-3864 
Fix #3864: DeviantArt fetch source data failure
 2018-09-04 13:42:01 -07:00
Albert Yi
8ec96f42f7 fix specs 2018-09-04 13:38:09 -07:00
Albert Yi
0529ec9ad0
Merge pull request #3862 from evazion/fix-3857 
Refactor searching text attributes
 2018-09-04 13:25:40 -07:00
Albert Yi
4a56f8d160 fixes #3856 for pixiv fanbox urls 2018-09-04 12:53:58 -07:00
evazion
e37844303d deviantart: take artist name from url when unavailable from API. 
In some cases we can't get the artist name from the API, either because
we can't do the API call because the url doesn't contain a deviation id,
or because the work is deleted:

* http://fc08.deviantart.net/files/f/2007/120/c/9/cool_like_me_by_47ness.jpg (work: http://fav.me/dwcohb)
* https://pre00.deviantart.net/423b/th/pre/i/2017/281/e/0/mindflayer_girl01_by_nickbeja-dbpxdt8.png (work: http://fav.me/dbpxd58)

Switch to taking the artist name from the url (when present) to deal
with these cases. Fixes the artist finder and the artist url normalizer
to work in this situation.
 2018-09-03 18:27:01 -05:00
evazion
8f87fb90d9 deviantart: handle urls without deviation ids (fix #3864) 
Some older URL formats don't contain the deviation id:

* http://fc08.deviantart.net/files/f/2007/120/c/9/Cool_Like_Me_by_47ness.jpg
* http://pre06.deviantart.net/8497/th/pre/f/2009/173/c/c/cc9686111dcffffffb5fcfaf0cf069fb.jpg

In these cases we can't make the API call. Fix failures due to not being
able to do API calls in this situation.

Also fix canonical_url to use the image_url when it contains the
deviation id, or the page_url when it doesn't.

Finally, fix page_url to use the url from the API instead of the raw url
given by the user, so that it's in a consistent form for canonical_url.
 2018-09-03 18:26:45 -05:00
evazion
316acead16 deviantart: fix error when uploading image belonging to deleted work. 2018-09-02 23:09:40 -05:00
evazion
2d1b1311d6 deviantart: fix sample urls not being rewritten to full size urls. 2018-09-02 23:09:29 -05:00
evazion
807c3dd5f4 deviantart: remove obsolete image sample rewrite rules. 
Remove rewrite rules for these types of sample urls:

* http://th00.deviantart.net/fs71/PRE/f/2014/065/3/b/goruto_by_xyelkiltrox-d797tit.png
* http://th04.deviantart.net/fs70/300W/f/2009/364/4/d/Alphes_Mimic___Rika_by_Juriesute.png

These URLs aren't served to users any more, and just stripping out "PRE"
or "200H" isn't sufficient to get the full size image. In general, an
api call is required to find the full size image url.
 2018-09-02 14:49:58 -05:00
evazion
b9ed676bfb deviantart: handle origin-orig.deviantart.net urls. 2018-09-02 13:57:15 -05:00
evazion
d693f01dde Fix #3859: Related tag and find artist don't run when fetch data fails. 
Fixes an exception in the artist finder caused by searching for a nil profile_url.
 2018-09-01 11:48:42 -05:00
evazion
d6235d6f9e nijie: add canonical url tests. 2018-08-31 23:23:15 -05:00
evazion
0eff095a3e Refactor searching text attributes. 
* Allow using ApplicationRecord#attribute_matches to search text attributes,
and standardize models on using this instead of duplicating code.

* Remove restrictions that limited wildcard searches to Builders only in various places.
 2018-08-31 19:50:46 -05:00
evazion
736c22c3ce pixiv: fix fetch source data failing on fanbox images. 
ref: https://danbooru.donmai.us/forum_posts/150000
 2018-08-31 00:21:03 -05:00
evazion
c689a161f6 pixiv: fix failure when normalizing pixiv stacc artist urls (#3856). 2018-08-30 19:24:44 -05:00
Albert Yi
9206a60760
Merge pull request #3852 from evazion/fix-twitter-direct-url 
Twitter: fix handling of direct image urls without a referer url.
 2018-08-29 17:32:54 -07:00
Albert Yi
48f2a79d13 fix artist url spec and bug with nicoseiga strategy not recognizing urls 2018-08-29 17:14:36 -07:00
evazion
a1044dbc19 twitter: fix handling of direct image urls without a referer url. 2018-08-29 17:14:57 -05:00
Albert Yi
66e413b540 Adds fuzzy string matching and prefix matching 2018-08-28 16:41:52 -07:00
Albert Yi
68c30961ac
Merge pull request #3805 from r888888888/refactor-sources 
Refactor sources
 2018-08-28 12:13:15 -07:00
evazion
3dafca9aec Fix #3842: Mods can demote other mods or admins. 2018-08-25 13:52:50 -05:00
evazion
2cdd022c37 Fix #3837: Remove expired bans. 2018-08-24 16:49:34 -05:00
Albert Yi
762dc3da24 Refactor sources 2018-08-24 12:10:51 -07:00
evazion
4f02c7f70a search: add status:modqueue, status:unmoderated metatags. 
* status:modqueue = ~status:pending ~status:flagged
* status:unmoderated = status:modqueue -user:self -approver:self -disapproval:any
 2018-08-23 14:48:39 -05:00
evazion
89c4fe150a search: add disapproval:<any|none|disinterest|poor_quality|breaks_rules> metatag. 2018-08-23 14:08:02 -05:00
evazion
9a0f37e359 Fix #3715: Provide any/none modifiers for pixiv: metatag 2018-08-22 00:25:38 -05:00
evazion
c504ad555d Fix #3822: Move favorites (while deleting) fails if user has also favorited destination 2018-08-19 00:54:17 -05:00
evazion
eaef616a01 Fixup tests for 03cf4c917. 2018-08-19 00:42:06 -05:00
evazion
fb91bbc6c5 Fix #3813: Favorite limit can be bypassed. 2018-08-12 14:22:08 -05:00
evazion
202527008f user.rb: drop unused add_favorite!, delete_favorite! methods. 
These methods were incorrect anyway, didn't upvote/downvote the post.
 2018-08-12 12:35:15 -05:00
Albert Yi
135b97d511 additional fixes for deviantart artist search (#3771) 2018-07-27 12:31:26 -07:00
Albert Yi
7753461f6f don't overwrite upload source with downloaded source 2018-07-26 18:34:00 -07:00
Albert Yi
77854349e5 testing 2018-07-26 18:11:19 -07:00
Albert Yi
9082ddf455 potential fix for #3783 2018-07-26 18:10:27 -07:00
Albert Yi
a0205be8b5 fixes #3771 2018-07-06 11:44:07 -07:00
Albert Yi
ff6d8910b5
Merge pull request #3746 from r888888888/post-flagger-report 
Post Flagger Report
 2018-07-05 16:54:50 -07:00
Albert Yi
6dfb78fca8 split out image cropping to width x height, add StorageManager::Match 2018-07-03 17:59:53 -07:00
Albert Yi
cdcd4d4014 Add additional post flag validation preventing targeting 2018-06-26 15:09:39 -07:00
Albert Yi
f4d5932a66 update dockerfiles 2018-06-25 17:29:01 -07:00
Albert Yi
64446d49e1 add image cropping support 2018-06-22 14:41:57 -07:00
Albert Yi
e551ff9b0c fix tests 2018-06-20 11:11:46 -07:00
Albert Yi
b0c2ddba8b update tests 2018-06-14 17:52:41 -07:00
Albert Yi
0e6c358701 add drag and drop file uploads w/async processing 
[skip ci]
 2018-06-14 17:52:41 -07:00
Albert Yi
fdd7582fb0 add support for upload preprocessing 2018-06-14 17:52:41 -07:00
Albert Yi
856637ace4 fix tests 2018-06-05 15:51:06 -07:00
Albert Yi
52de1fb981 clear artist urls before saving url string (fixes #3731) 2018-06-04 17:37:43 -07:00
r888888888
ca842cc6d9 fixes #3727 2018-05-26 12:58:19 -07:00
Albert Yi
5ae37597cd fixes #3728 2018-05-25 13:24:49 -07:00
Albert Yi
b7c4df2df0 fixes #3724 2018-05-24 10:24:14 -07:00
Albert Yi
fcd80b6043 Artist urls can be prepended with a '-' to mark is inactive (#3388) 2018-05-16 16:04:05 -07:00