Base Module¶
tinyCrawl.base.base module¶
Created on 2021/1/7 13:36
@author: Irvinfaith
@email: Irvinfaith@hotmail.com
-
class
tinyCrawl.base.base.BaseCrawl(iter_url, iter_num_range, thread_num)¶ 基类：
object-
crawl(*args, **kwargs)¶ Crawling task function, need to be implemented.
-
main()¶ Main function to excute crawl and sink function.
-
run(obj)¶ Adding a crawling function into the class
- 参数
obj – An crawling function object
-
sink()¶ Output the crawling result, need to be implemented.
-
tinyCrawl.base.base_container module¶
Created on 2021/1/10 14:44
@author: Irvinfaith
@email: Irvinfaith@hotmail.com
-
class
tinyCrawl.base.base_container.Container(*args, **kwargs)¶ 基类：
list-
classmethod
clear()¶ Clear the data stored in the container.
-
container= {}¶
-
classmethod
-
class
tinyCrawl.base.base_container.RowContainer(*args, **kwargs)¶