[137.79 GB] [Misc] Furaffinity.net SiteRip Part 1 of 72 (-----. Sora .----- ... Aeolos) [Furry, Yiff] [PNG, JPG, GIF, SWF etc.]

Furaffinity.net SiteRip / Rip site Furaffinity.net
Publisher site: https://www.furaffinity.net
Distribution type: Misc
Genre: Furry, Yiff
Page Resolution: from tiny to a giant
Number of pages: 24500015 pcs., 472624 artists
Format: JPG, PNG, GIF, MP3, SWF, TXT, DOC, PDF, ODT, ...
Description: Full Site Site Furaffinity.net at 06.01. 2021 (before post №40000000).
sorted by artists.
FUR Affinity - the largest online gallery and library of stories dedicated to anthropomorphic animals. Created in 2005. Along with Sofurry and Inkbunny enters the "big troika".
Because of the absolutely titanic size of the distribution - both from the point of view of volume and in terms of the number of files, it is published divided by 72 volumes. For the same reasone, as well as due to the lack of a normal tag system, the separation of orientation this time will not ("loved by selfless nude - be kind to get a portion of eggs" © some Four with a joyreactor).
To make the number of parts small Slisky was sane, it was decided to break the distribution on parts not by letters, but in arbitrary places - as in the Big Soviet Encyclopedia. This part contains drawings of artists whose names (in alphabetical order) are between "-----. Sora .-----" and "Aeolos".
Under the "alphabetic order" is understood as the following: - ,. , 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, [,], ^, `, a, b, c, d, e, f, g, h, i, j, k , L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, V, W, X, Y, Z, ~.
The remaining parts
in the last distribution are examples to all volumes. If you read these lines in 2030+ and fAstpic once again removed them, you can swing from that distribution folder "Design".
Add. Information:
Metadata
As before, in parallel with the creation of the database of downloaded files, I built both the database of all other data available on the site (author's descriptions, tags, a list of comments, etc.). It turned out not so much - "only" 50 is 50, "and I planned to add this database to distribution, so that a full-fledged backup of the site was turned out.
That's just, to my shame, for all the past years I have so and Hands did not reach the study of SQL, so this database has a self-written format, and when it would be published, I would have to (again, as before) to attach CPP sources to it with a description of this very format. I'm tired of this gang, and this time I firmly decided to master SQL, envelopeIt's your crutch database into it, and put it in the distribution already in a normal form ... But, alas, I did not have time.
Somehow later I'll post down and post it somewhere.
Duplicates of files
Unlike E621 .NET, where any downloadable file is renamed into its MD5-hash and therefore duplicas (at least binary) are not possible in principle, on the FA the same file can be downloaded as much as possible.
Before packing distribution in the archives, all binary Duplicas (about 1.6 million files) were removed. In order not to lose the connection, when you delete each such dub, it was recorded in a special Dupes.txt file, lying in the archive of the appropriate user (the file from whose gallery was deleted). File syntax Such:
Post-duplicate \ T File-duplicate name \ N Original post \ T Name of the original author\ T Name of the original file (renamed if it was renamed; see below) \ N \ N
As you can see, you can easily paint these files by any programming language (well, either just open in notepad and see what users you need To climb the additive and what files you need to search).
, taking into account the fact that a significant part of the FA users does not draw anything and only orders work, this removal of duplicates led to "empty" archives in which there is nothing but this file itself Dupes (such 4249 pcs).
It is worth mentioning that the common reason for the appearance of the double looks like this:
- the customer orders the artist drawing;
- the artist performs the order and posts in his gallery;
- Customer, Do not soar, downloads the drawing with FA and loads it back on FA,But already in your gallery.
When downloading to the file name, a random number is added to the name, and then the name of the loading one. In the situation described above, two users are provided in the file name - first the customer, then the artist; Therefore, if you wish, you can guess which of the two doubles should be removed.
The problem is that a good thought comes after, so the choice of which of duplicates "spare" was not deterministic. (True, the posts were processed in a more or less chronological order, so that the highest of the oldest files were in the distribution - just the same those that are usually downloaded by the artists themselves).
File names
Oh ... well, with this Treshak .
Let's start with the fact that for a typical artist in the order of things to maintain a drawn picture under an incredible expansion. for instance, File has a PNG extension, and the file itself has a JPG format.
on FA such files - millions (only for this particular pair "PNG → JPG" - 3.2 million). Why so - I do not know. Apparently, they are used to that if in the "Save how" file name of the file with the extension, the editor itself guesses the use of the desired format - and then switched to some other editor in which the format must be specified manually.
Anyway, This is a problem, because many image views will refuse such files to open. To save you from the headache, before packing the MIME files of each file, each file was checked with its extension, and in case of incomprehension, the latter corrected.
To - again - not losing a connection, each such an act of renaming was recorded in a special renames.txt file, le.Archive in the archive of the corresponding author. Syntax of the file:
Post \ N The original file name \ n The renamed file name \ N \ n
just in case it was done only for three formats - JPEG, PNG and GIF, - because only FA for 15 For years, so much of all garbage was loaded that only the list of MIME detected takes several pages. But MIME is not always a definitely faithful verdict; For example, completely and next to DOCX files having MIME "Application / Zip" instead of "application / vnd.openxmlformats-officeDocument.wordProcessingml.document".
Going further.
When you download the file from the site, the browser creates The file name (under which you need to save the downloaded file to the disk) based on the link in which this very file was downloaded.
Here just to linksThere are strict requirements - in particular, many characters are not allowed in them and should be replaced by special "interest codes".
So. The FA web server encodes some reserved characters in some posts.
For this reason, it is impossible to start the name of the file with the programmate, analyzing the downloaded HTML, it is impossible. You can either
- apply decryption, and get from the link "//d.facdn.net/art/bigwolfbebad/110755457/wolfgangsketch # 1.jpg" (post 13250) The name "WolfGangSketch" instead of "WolfGangSketch # 1.jpg" ; Either
- not apply, and receive from reference "//d.facdn.net/art/keto/1134274040/1134274040.keto.itisn%27Tiswrong.jpg" (post 5361) Name "1134274040.keto.itisn% 27tiswrong .jpg "Instead of" 1134274040.keto.itisn'tiswrong.jpg ".
Sreeks his teeth, I still chose the first varIANT, and as a result, received about 18 thousand chopped titles. (Expansion to them, however, it was still automatically added, so you can live).
Next.
Fa allows you to replace the downloaded file. Very often it leads to the fact that the link to this very "corrected" file is irreversibly urged (usually the file extension is represented somewhere in the middle of the link, giving something like "//d.facdn.netpdf/art/..."). The post becomes a "broken" (when you try to open it on the site you will see a 120x120 wallpaper for a dental pain with the inscription "Image Not Found").
But sometimes a miracle occurs, and the download of the changed file works. When stars came out in the sky and such a change fell between the two of my ripping sessions, and this post fell into both sessions, then gotForce Major: The algorithm of rejection of the double gave failure (the contents of the file is different), and the corrected file was added to the ZIP archive along with the previous version.
In principle, there is nothing terrible in it - the zip format specification allows inside the same archive How many files with identical names, and any archiver will work with such an archive without problems. Problems will begin when trying to unpack all the contents into one folder. Most likely, the folder simply will remain the latest version (which, in general, and necessary). In addition, such force majeures were only about a hundred.
Empty files
When trying to download something from the server, it sometimes sends a file with a size of 0 bytes. The problem is that this can be caused by three different reasons:
- the server is a normal file, but something is brokenelk on my side (the connection was cut off, for example);
- the server lies a broken file (most of these files appeared in the summer of 2008, when the server fell and buried everything that was ever loaded to the site - the admin was then blocked, What could, within a month; a clear example of what the backup copies are needed like this very distribution);
- the author was deliberately uploaded to the FA an empty file.
It is impossible to distinguish these options to these options, so I had to consider each manually (They got about 500). To assign a width "empty server" viewed file, I recorded the only byte byte to it (~). If the file had an extension TXT, then instead, a somewhat more sensible plug "" was recorded.
The Great Randa decided that the first file with TIldoe will be processed "1368952479.tomslove_deni2.jpg" from the author Tomslove (post number 10632532), and the first text file - "1368782815.angel-blackwolf_nuevo_documento_de_texto.txt" by Angel-Blackwolf (post №10617741). Then in hundreds of archives of other authors in the Dupes.txt files there are inappropriate links to these two files. So if you see one of them - do not hurry to climb into the appropriate archives, these are just empty files.
Sorry, it happened. It happens when it is necessary to finish the "live" program, which is already grinding the terabytes of files for a month.
Linux users
The zip format has a generic injury - it does not store the encoding used for file names. Windows archivers treat this injury with fortune telling on the coffee grounds,When the archive is opening, he survitically determine the desired encoding. In * n? The same file name is traditionally considered not a text string, but byte a set, so that the developers of most utilities for working with archives simply do not see any problem here - well, all the files have turned out to be called krakoyabra, and what? Maybe they were so sparkled - they had the right.
In short, in this hand, all file names in archives are in the UTF-8 encoding. The usual Unzip (1) and File-Roller when unpacking such archives will give you the mentioned krakoyar. Solution - to supply P7zip and instead of "UNZIP" use "7z x".
From myself
If you have to rephrase someone's a long-visited commentary, digging in the furry fandome - this is the search for a treasure chest in the sewer system. You clock wrap upHenn in shit, sometimes falling down with his head, but when they finally bother to another hidden pearl - you feel that it was worth it.
If you do it regularly, a certain protective crust is gradually growing, like corn corollas. You become stronger.
There are masterpieces of painting, worthy of hanging in art galleries - they are lying under the deposits of Krivolapa Mazni, which a 12-year-old child would have been awarded. Here there are Shakespeare Plots, turning the soul inside out - densely agonated fetishes, from which they pull to bleak.
Wear a rhbz costume and asbestos crags, take a scandalous shovel - and on the road.
"Art Should Comfort the Disturbed and Disturb The Comfortable. »
- Cesar A. Cruz