If you have seen my previous article, my friends must know that I have built an e-book search engine. There are less than 10,000 books in it, but they have not been made public. The main reason is the copyright issue. It is not illegal for me to hold these documents privately, as long as I don't say loudly, no one knows that I have these documents, but * publicly disseminated, *public sharing is suspected of breaking the law. And I don't want to let this information bury on the Internet, I want it to continue to shine, so I decided to make the search engine public. But before that I did some ** tiny work to prevent me from getting into trouble in the future because of this decision.

The book is the ladder of the progress of mankind. ——Gorky

Then the e-book is the elevator. ——SaltyLeo

# Copyright law

Since the British Parliament passed the world's first copyright law, the Queen Anne Act, in 1709, the core of the 300-year copyright protection in the printing era was the right to copy, but in the Internet age, the concept of reproduction has undergone subversive changes, and network communication. At the same time, the completion of the copy, the copy right and the right to spread become a fit. Unlike paper books, most e-books cannot be resold, transferred, or loaned.

👇Screenshot from "Kindle Store Terms of Use" 。

I personally respect the copyright law. I also bought a lot of paper books at home, and there is no place to put them. The advantage of e-books is that they don't need to take up space to store a large number of books, but now the e-book mode is equivalent to * borrowing*, paying for a copy, can not move around, can only be seen on the only device or app. If you want to share it with someone, lend it to someone else in the age of paper books. What about e-books? You have to give the account to someone else to share it with others. I don't agree with this.

Therefore, it is natural to create pirated e-books, use illegal means, or directly OCR original paper books to produce e-books in public format. The pirated e-books with EPUB, MOBI, and AZW3 formats are gradually flooding the market because it is extremely easy. Sharing communication has caused huge economic losses to the publishing house. Therefore, the publishing house has implemented a high-pressure policy for such pirated e-book websites. As long as we find some clues, it will be madly attacked, so the e-book websites we know are disappearing a little. . For example: booksetme, small bookstore, etc.

The reason for this is that the webmaster of the website is not aware of the network security and is not doing a good job of anonymity. Secondly, the legal awareness is weak. I don't want to take their old path > Build a website> >Share data`, which is undoubtedly a self-digging grave. Because the first step to build a website to provide data violates the rights of the person to copyright, and the domain name resolved by the server in China needs to be filed. The domain name after filing is like an ID card. Find your personal information and even have a half-length photo.

So the solution I chose was a search engine that searched for the corresponding data based on the input and provided the corresponding off-site link, which was not stored on my server, including search engine index data.

According to China's Information Network Communication Rights Protection Regulations》Article 14 (excerpt)：

A network service provider that provides information storage space or provides search and link services. The rights holders believe that the works, performances, audio and video products involved in their services violate their own information network transmission rights or are deleted and changed their rights management electronics. For information, the network service provider may submit a written notice requesting the network service provider to delete the work, performance, audio and video product, or disconnect the work, performance, audio and video products. The notice should contain the following: (1) the name (name), contact information and address of the right holder; (2) requesting the deletion or disconnection of the name and network address of the infringing works, performances, audio and video products; (3) The preliminary certification materials that constitute the infringement. The right holder shall be responsible for the authenticity of the notice.

Suppose the link content I provided infringes the right holder's 'Information Network Copy Right', please send a DMCA notice to saltyleo@tstrs.me, please fill in the corresponding content in standard format, need to include the fourteenth The three elements of the provisions.

Article 15 (Excerpt):

After receiving the notice from the right holder, the network service provider shall immediately delete the allegedly infringing works, performances, audio and video products, or disconnect the allegedly infringing works, performances, audio and video products, and at the same time forward the notice. The service object of works, performances, audio and video products; if the network address of the service object is unknown and cannot be transferred, the contents of the notice shall be announced on the information network at the same time.

Article 23 (Excerpt):

... the network service provider provides a search or link service for the client, and after receiving the notice from the right holder, disconnecting the infringing works, performances, audio and video products in accordance with the provisions of these Regulations, shall not be liable for compensation; ...

Therefore, in accordance with the above provisions, I am in compliance with the Notice-Delete procedure. Will not be liable for compensation.

# IPFS(InterPlanetary File System)

InterPlanetary File System is a network transport protocol designed to create persistent and distributed storage and shared files. It is a content-addressable peer-to-peer hypermedia distribution protocol. The nodes in the IPFS network will form a distributed file system.

In order to ensure that I am not listed as the object of the complaint, I do not intend to put the file on my own server, I chose InterPlanetary File System as a storage service, the reliability is not high, but robust Very strong, as long as there are nodes in the world, you can find the file.

Ordinary http link, such as the logo of Douban:

https://img3.doubanio.com/f/frodo/144e6fb7d96701944e7dbb1a9bad51bdb1debe29/pics/app/logo.png 

Just like this👆, the file that itself has any changes to the characters will change, because http is a `path-oriented' protocol.

IPFS is a file-oriented protocol. It does not need to know the path. It only needs to know the hash of the file. It will automatically search for the hash in the IPFS network, find it, and render it.

The IPFS link looks like this:

https://gateway.i-book.in/ipfs/QmNY7V1MFgvo1ihdDdFYT4UUGm4dz8HVwqXXL1DdxYePLB

Open it like this:

The part of gateway.i-book.in is the gateway. Because this protocol has not been developed, it needs to access this special peer-to-peer network through the gateway. After the browser and the service provider support, the link will be simplified to:

ipfs://QmNY7V1MFgvo1ihdDdFYT4UUGm4dz8HVwqXXL1DdxYePLB
or
fs://QmNY7V1MFgvo1ihdDdFYT4UUGm4dz8HVwqXXL1DdxYePLB

And the most amazing thing is that you can change the different gateways to get this file👇:

You can use the gateway of ipfs.io：
https://ipfs.io/ipfs/QmNY7V1MFgvo1ihdDdFYT4UUGm4dz8HVwqXXL1DdxYePLB
Or Cloudflare's gateway：
https://cloudflare-ipfs.com/ipfs/QmNY7V1MFgvo1ihdDdFYT4UUGm4dz8HVwqXXL1DdxYePLB

The files that are ultimately accessed are the same one. Look carefully at the link.：

This is where the agreement is most appealing to me, node as the lowest level of existence, cluster is based on nodes, gateway is the result of the current protocol is not developed and some details are compromised. Hash is the soul of this distributed network. As long as there is a node in the entire network that stores the file represented by this hash, the nodes on the whole network can access this file, the difference is only the access speed.

Of course, some files disappear from the network even if the source node does not go out, but you can use the pin function to pin the file of the remote node locally. Even if the source node is closed, the file can still be transmitted on the network.

I will not repeat the details of this agreement here. I have the ability to go directly to IPFS.io to see the white paper.

note! ! ! All IPFS unofficial communities are scammers. All IPFS mining is a liar (except Filecoin), because this is a pseudo-requirement. Will you pay for the storage of some unimportant files? I do not know. Even if I pay for the building node, I am not willing to pay the data to others, just like when the bt protocol was very popular, the PT station was created to keep the bt seed availability. How much is still online now? IPFS is now in very good shape. You can use public IPFS networks to spread data, or you can build a private IPFS network, as long as you want.

Because i-book.in does not store any data itself, it just responds to forwarding so I think this should be the current neutral scheme.

For this program I did some tiny work:

The i-book.in page uses even the search service provided by Algolia. The gateway.i-book.in gateway uses the IPFS. The front-end uses CloudFlare CDN service. All file storage uses IPFS, includes pictures.

The final result is completely non-stick, I have nothing to do with the data, all the data of the search engine comes from other websites, using crawlers to crawl the integrated information to provide search, I am just the porter of the Internet.

# Postscript

Here I would like to thank [LoreFree] (https://ebook.lorefree.com/), which gave me inspiration. Their backend file storage system is based on IPFS technology, but the combination with EOS is the wrong decision. Any free sharing is unclear as long as it is related to money. Recently, their DAU has fallen to double digits. I don't know how long they can last. The technology stack is quite perfect, anti-crawl is very strict, even for anti-climbing can open the verification code forever error mode (just if you enter any verification code is wrong, this is my personal guess, it is possible Is the internal system error), but it does not matter, the next one is it, LoreFree's database total of 66,511 books (2019-08-30 data), according to the average of three books, one book total of about 10M. It is estimated that there is also about 700G data, um, really fragrant.

Although I have written(quibble) so much here, I guess there will be someone who will tease me, or prepare to write DMCA, or prepare to contact the publisher to earn extra money, because the ID of SaltyLeo is not completely anonymous. A little bit of relationship to Tencent Cloud check tstrs.me domain name registration information can find me, or social workers? Therefore, I am a good young man who is law-abiding and patriotic. As soon as I receive the notice, I will delete it immediately and act in accordance with the law.

The correct setting of nginx forwarding is really tired, debugging iptables is also annoying. When I finish it, I will go to GitHub ~~open source~~(friends suggest me not to do this, so not open source, sorry)index database.

On how to set up an IPFS node, how to build a cluster, etc., I will update it if I have time. Oh, yes, the University of Adelaide’s library also has an e-book sharing project, but it’s all publicly copyrighted. It’s a very old book, here: https://ebooks.adelaide.edu.au/, the next target for crawling is it.

This search engine is nearing the full version, and it is difficult to crawl the index data and expand the Algolia.

I don't intend to make a profit, but I am curious about the technology implementation. If there is any bad sign, I will go offline directly.

Sometimes it is quite sensible to look at the old news. Compared with the current style of writing, this kind of focus event itself is not concise and powerful, and it is more concise and powerful to use exaggerated metaphors to attract attention.《电子书是人类进步的"电梯"? 纸本阅读速度更快》

PS: The "Regulations on the Protection of Information Network Transmission Rights" written above is a legal provision in China. I simply translated it. It may not be very accurate. The specific content can be viewed in this link: http://www.gov.cn/zwgk/2006-05/29/content_294000.htm

-EOF-