Opitions¶
Fulmar is easy to use.
Note: Redis is necessary, so make sure you have installed it.
–help¶
You can get help
, just run:
fulmar --help
You will see:
Usage: fulmar [OPTIONS] COMMAND [ARGS]...
A crawler system.
Options:
-c, --config TEXT a yaml file with default config. [default:
/fulmar/fulmar/config.yml]
--redis TEXT redis address, e.g, 'redis://127.0.0.1:6379/0'.
--mongodb TEXT mongodb address, e.g, 'mongodb://localhost:27017/'.
--phantomjs-proxy TEXT phantomjs proxy ip:port.
--logging-config TEXT logging config file for built-in python logging
module [default: /fulmar/fulmar/logging.conf]
--version Show the version and exit.
--help Show this message and exit.
Commands:
all Start scheduler and worker, also run...
crontab Crontab infos and operations.
delete_project Delete a project.
phantomjs Run phantomjs if phantomjs is installed.
scheduler Run Scheduler.
show_projects Show projects.
start_project Start a project.
stop_project Stop a project.
update_project Update a project.
worker Run Worker.
–config¶
Config file is a YAML file with config values for global options or subcommands. Fulmar has a default config file, the content is:
redis:
url: redis://127.0.0.1:6379/0
mongodb:
url: mongodb://localhost:27017/
worker:
async: true
poolsize: 300
timeout: 180
If you run fulmar without any paramtets or config file, fulmar will use this default configuration. You can write your own config file, and use it just like:
fulmar --config=your-config-file all
–mongodb¶
MongoDB address.
–phantomjs-proxy¶
phantomjs proxy ip:port. If you set it, it means you have already run phantomjs. So fulmar will not try to run a new phantomjs, instead just use this one.
–logging-config¶
Log config file. Fulmar use logging. If you want to change the default log behavior, you can write you own log file, reference: configuration-file-format
–version¶
Show fulmar version.