Commit Graph

447 Commits

Author SHA1 Message Date
eXtremeSHOK
8eb05d1450
Fix for large Mailboxes to avoid timeouts 2
Missing ' (sorry my editor keeps correcting the '" to " )
2018-02-14 14:02:56 +02:00
eXtremeSHOK
a37a8e3b2a
Fix for large Mailboxes to avoid timeouts
** I agree to the code of conduct and the contributory guidelines **

Tested with 80+GB single inbox via imap

``` --nofoldersizes --skipsize --fast ``` file and folder sizes are only used for statistics, which are completely useless. Before the actual data is transfered the message and folder sizes need to be calculated on a 200 000+ message inbox, this will almost never complete.

``` --buffersize 8192000 ``` sets the io buffer to 8mb, the default buffer is 4kbyte, this is speeds up syncs.

```  --skipheader 'X-*' ``` X headers vary wildly from system to system and do not ensure message uniqueness, they are not needed.

```  --split1 3000 --split2 3000 ```    split the requests in several parts on the server, 3000 is the number of messages handled per request.

``` --fastio1 --fastio2 ``` use fastio
2018-02-14 13:27:30 +02:00
Phoenix Eve Aspacio
42868b1710
Merge pull request #1037 from klausenbusk/empty
Remove "empty" folders
2018-02-14 08:15:13 +08:00
Kristian Klausen
a0cdc1e4ff Remove "empty" folders
There seems to be no reason for this empty folders.
2018-02-13 23:45:49 +01:00
André Peters
66a3df16ed [Netfilter] Fixes empty f2b options 2018-02-09 10:32:12 +01:00
André Peters
83094a773e [Watchdog] Use socket to pipe to Rspamd 2018-02-08 22:57:29 +01:00
André Peters
004d262e41 [Rspamd] Create missing password include as placeholder 2018-02-08 22:57:17 +01:00
André Peters
ac4982d706 [Dovecot] Use socket to pipe to Rspamd 2018-02-08 22:56:43 +01:00
André Peters
a2ca550aa0 [Dovecot] Use socket instead of worker IP 2018-02-08 22:29:43 +01:00
André Peters
e3854a8037 [Dockerapi] Fixes recent chang in exec_run return 2018-02-08 22:29:06 +01:00
root
fae542534d Merge remote-tracking branch 'origin/master' into dev 2018-02-08 19:54:14 +01:00
Joshua Hesketh
2fc13bdfbe Fix imapsync for large mailboxes
Where there is a lot of folders in a mailbox the output may be larger
than TEXT will allow (64KiB). Instead use MEDIUMTEXT (16MiB) which
should be ample.

Additionally check that a sync isn't already running before starting
a new one.

Fixes: #1011
2018-02-08 15:24:04 +11:00
andre.peters
cf1e46723a [Dovecot] Fixes CVE-2017-15132 - take 2 2018-02-01 23:37:10 +01:00
andre.peters
3f9e0a8023 [Dovecot] Fixes CVE-2017-15132 - take 2 2018-02-01 23:36:06 +01:00
andre.peters
ae4ccd4d17 [Dovecot] Fixes CVE-2017-15132 - take 2 2018-02-01 23:35:55 +01:00
andre.peters
6ebcd00521 [Dovecot] Fixes CVE-2017-15132 2018-02-01 22:43:28 +01:00
andre.peters
c5f9b065f6 [Dovecot] Fixes CVE-2017-15132 2018-02-01 22:27:48 +01:00
andre.peters
a0d9efba00 [PHP-FPM] Remove old migration scripts 2018-02-01 13:40:59 +01:00
andre.peters
f4ae354c0c [SOGo] Do not try to use foreign mailboxes as alias 2018-02-01 13:40:13 +01:00
andre.peters
38a819771b [Netfilter] Rename fail2ban to netfilter, use iptables-python 2018-02-01 13:39:27 +01:00
andre.peters
0773448b35 [Dockerapi] Fix jsonify output 2018-02-01 13:38:42 +01:00
André Peters
912ba9b4ff
Fixes #979 2018-01-29 14:42:51 +01:00
Michael Kuron
c30448c4d8 Merge branch 'master' of https://github.com/andryyy/mailcow-dockerized into recipient_map
Conflicts:
	data/web/inc/init_db.inc.php
2018-01-27 17:22:08 +01:00
andre.peters
7433b6dc91 [ClamAV] Build from source, fix bugs...
[Compose] New clamd-mailcow image
2018-01-27 10:26:12 +01:00
andre.peters
04f9d74339 [Web, Dovecot] Add new options to imapsync, other minor changes to forms, partly fixes #955 2018-01-24 12:59:11 +01:00
andre.peters
1aaa5682b4 [Fail2ban] Allow to set subnet size for banned networks 2018-01-24 09:11:33 +01:00
andre.peters
f2f4dabce4 [Postfix] postconf wrapper for correct config location, fixes #949 2018-01-24 09:10:43 +01:00
andre.peters
46aafff627 [ClamAV] Outsource config 2018-01-24 08:40:13 +01:00
Michael Kuron
e86565e283 Expose Postfix's recipient_canonical_maps through web UI 2018-01-23 20:02:31 +01:00
andre.peters
a7a7b3f3fd [Postfix] Use name instead of IP 2018-01-21 15:01:51 +01:00
andre.peters
d24bb16947 [Watchdog] Check PHP-FPM port 9000 and 9001 2018-01-21 15:01:35 +01:00
andre.peters
737c41379f [PHP-FPM] Move opcache config to local config file, define new PECL versions 2018-01-21 15:01:02 +01:00
andre.peters
83a21259f7 [Rspamd] Use names instead of IPs 2018-01-21 15:00:05 +01:00
andre.peters
08c8976a95 [SOGo] Show shared aliases and "allow to send as" addresses as FROM fields in SOGo 2018-01-21 14:58:16 +01:00
andre.peters
8419266678 [Web] Important fixes for quarantaine; other minor changes 2018-01-17 15:23:33 +01:00
andre.peters
09b6c20bad [DockerAPI] Hide stderr from rspamadm output 2018-01-14 18:44:06 +01:00
andre.peters
5d5d36fc60 [Dovecot] Revert to 2.2 to fix various errors 2018-01-14 10:44:06 +01:00
andre.peters
38aaeeb30b [Dovecot] Dovecot 2.3 from git for temp fixes
[ACME] Use -a switch (thanks to jas8522!)
2018-01-14 10:34:06 +01:00
andre.peters
42923698fd [Dovcot] Fixes for v2.3 2018-01-13 22:49:50 +01:00
andre.peters
ceef6d6fd3 [Dovecot] Dovecot 2.3, Pigeonhole 0.5
[Watchdog] Fix Dovecot health check
2018-01-09 11:15:52 +01:00
andre.peters
b7a23a28fd Merge branch 'dev' of https://github.com/mailcow/mailcow-dockerized into dev 2018-01-08 22:01:39 +01:00
andre.peters
1e9cae9084 [Compose] New images with LOG_LINES
[Update] Do not add empty line in each loop
2018-01-08 22:00:54 +01:00
Michael Kuron
cde250a989 Properly wait for MySQL to come up before starting SOGo 2018-01-08 19:56:44 +01:00
andre.peters
db032af698 [ACME] Fix script 2017-12-31 17:17:46 +01:00
andre.peters
8d56534e76 [Postfix] Don't try to authenticate to relayhosts without username, fixes #725 2017-12-25 10:18:49 +01:00
Oratorian
236e4d4a36
See Issue #826
Fixes dockerapi-mailcow_1 | raise TypeError('port must be an integer')

Containers are now restarting.
2017-12-18 16:41:04 +01:00
andre.peters
52f4f850cf [Postfix] Fix missing authentication data for relayhosts when sender domain is alias domain 2017-12-17 17:45:12 +01:00
andre.peters
a771d66889 [Dovecot] Update imapsync 2017-12-17 17:45:05 +01:00
andre.peters
2994b94b6c [Docker API] Remove logs, remove env var for compose project name 2017-12-11 09:43:01 +01:00
andre.peters
6c67b9df82 Replace name by IP, remove unused tables 2017-12-09 22:30:18 +01:00
andre.peters
2519738094 Various changes... 2017-12-09 13:15:24 +01:00
Peter Schiffer
7d6fc8e6b6 [Dockerapi] Auto detect version of docker server
Some older versions of docker need specific version of client api to be able to
communicate. This change allows automatically detect and set version of API to
match server version of API.

Fixes #765
2017-12-01 23:41:37 +01:00
André
ade4b9e7ae [Postfix, Web] Feature: BCC maps 2017-11-19 15:13:43 +01:00
André
5211ab10ed [ACME] Fix license issue 2017-11-16 14:57:17 +01:00
André
c45ea5433f [Rspamd] Base on xenial to fix mime types problems 2017-11-15 12:29:18 +01:00
André
fe845ee56d [Fail2ban] Fix fail2ban container 2017-11-14 19:50:20 +01:00
André
84a7a1a2e7 [Compose] New images, Nginx checks for SOGo before bootstrapping
[PHP-FPM] Some more modules (primarily for Horde)
[Fail2ban] Do not log matches of local and private ips
[Watchdog] Some changes in log system for further processing (wip)
[ACME] Fixes #745
2017-11-14 10:44:22 +01:00
André
60e97503f7 [Web, Dovecot] Show wether a sync job is running, validate min max input attr and validate these values 2017-11-08 11:07:32 +01:00
André
a36a8828c2 [Dovecot] Specify supervisord user 2017-11-05 12:19:18 +01:00
André
1ef10f1358 [PHP-FPM] Include net_sieve, test removal of usr/src/php for size 2017-11-03 20:27:43 +01:00
André
a9f64a3472 [Dockerapi] Return answers in json 2017-11-03 20:26:09 +01:00
André
b32e5adcc5 [Dovecot] sieve_before/after maps in sql, changed dict names 2017-11-03 20:25:38 +01:00
André
1e9bc49f2c [Rspamd] Echo dummy for fowardingshosts map; Use higher map reading interval;
[Dockerapi] Exit on sigterm;
[Watchdog] Wait for dockerapi-mailcow to be online
2017-10-27 11:22:39 +02:00
André
04cb033f0a [PHP-FPM] Add imagemagic 2017-10-21 10:10:27 +02:00
André
a110e2ea0f [ACME] Fix detection of orphaned SANs and add tini 2017-10-21 10:08:20 +02:00
André
6b6470fe54 [Rspamd] Use tini 2017-10-15 09:31:19 +02:00
André
ac413058c1 [Watchdog] Fix kill -0 check, use tini and send kill request to tini instead of parent pid, sleep longer 2017-10-14 23:26:08 +02:00
André
c5dd30b058 [ClamAV] Use tini, check if background procs are running, use pipe to output to stdout 2017-10-14 23:25:29 +02:00
andryyy
c5054ae7ed [Watchdog] Ignore null name in jq
[Nginx] Merge sites
[Scripts] Nextcloud helper script (testing!)
2017-10-11 22:56:22 +02:00
andryyy
7c46d6548b [Dovecot] Ignore watchdog IP in logs, filter by syslog-ng 2017-10-11 11:23:20 +02:00
andryyy
e107cbef5e [Postfix] Fix sending as alias, when alias is in alias domains, cleanup 2017-10-11 11:22:52 +02:00
andryyy
2862b43c81 [Watchdog] Fix watchdog to fit non-exposed PHP 2017-10-09 15:54:54 +02:00
andryyy
6110ac386f [SOGo] Use official nightly; [PHP-FPM] Fix expose=off 2017-10-09 15:45:48 +02:00
andryyy
72995ff98e [PHP-FPM] Include more modules for upcoming features and Nextcloud support, drop ro flag; [Watchdog] Some fixes and changes 2017-10-08 22:47:52 +02:00
andryyy
ef9953898c [ACME, Watchdog, DockerAPI] Use only limited Docker API 2017-10-06 13:32:49 +02:00
andryyy
3ae0b16845 [Web, DockerAPI] Be more like official Docker API 2017-10-06 10:20:40 +02:00
andryyy
fc18d153cd [Compose, DockerAPI, Web, Watchdog] Watchdog may send notification mails (todo: docs), DockerAPI via Flesk for limited access 2017-10-05 23:38:33 +02:00
andryyy
9860d44d04 [Watchdog] Do also log errors to Redis if availble 2017-10-04 23:18:51 +02:00
andryyy
2dc8306b69 [Postfix] Remove old socket 2017-10-04 23:15:26 +02:00
andryyy
82ac5fa063 [SOGo] Remove supervisord API 2017-10-04 13:04:35 +02:00
andryyy
9b4ed6b21c [PHP-FPM] Include Docker api for better SOGo status handling and future changes 2017-10-04 13:04:15 +02:00
andryyy
da987e5b48 [Postfix] Forgot 'not' in filter 2017-10-03 16:54:18 +02:00
andryyy
c59d03fcb3 [Watchdog] Skip when use_watchdog=n 2017-10-03 12:07:48 +02:00
andryyy
68d7fa1504 [Watchdog] Skip when use_watchdog=n 2017-10-03 12:05:38 +02:00
Michael Kuron
752a571607 Merge pull request #638 from mkuron/acme
ACME needs to wait for MySQL to be ready
2017-09-28 12:58:07 +02:00
Michael Kuron
ae79445ec0 ACME needs to wait for MySQL to be ready 2017-09-27 19:48:25 +02:00
andryyy
337c9e350e [Watchdog] Reset diff, new image 2017-09-22 16:40:02 +02:00
andryyy
62524150d2 [ACME] Add timestamps, check if acme account key is valid
[Postfix] Ignore local0
[Watchdog] Add Rspamd checks
2017-09-21 19:30:09 +02:00
andryyy
ab850dc901 [ACME] Detect and fix invalid registration 2017-09-21 09:46:09 +02:00
andryyy
41d2a16571 [Watchdog] Script was not executable 2017-09-20 23:36:04 +02:00
andryyy
f511cb0f63 [Watchdog] More fixes and or changes 2017-09-20 23:24:56 +02:00
andryyy
fd1955edca [Fail2ban] Add variable name 2017-09-20 23:24:39 +02:00
andryyy
b6e84fac3a Sleep instead of stopping containers to prevent restarts 2017-09-20 12:50:50 +02:00
andryyy
df5c79c3f1 Fixes for watchdog! 2017-09-20 12:27:24 +02:00
andryyy
e70d5b9206 Fix watchdog 2017-09-20 11:05:23 +02:00
andryyy
a8fb1d3f4f Add experimental watchdog 2017-09-20 10:56:49 +02:00
andryyy
d0c0cd4992 [Rspamd] Fix user settings... 2017-09-16 23:27:13 +02:00
andryyy
1b974bc8d1 [Compose] New images 2017-09-16 23:05:33 +02:00
andryyy
2b97305f6d [ACME] Sleep, don't exit 2017-09-16 13:17:48 +02:00
andryyy
762f18e913 [Clamd] Use Dockerds own init system 2017-09-14 23:13:24 +02:00
andryyy
eeaa48a729 [PHP-FPM] Use valid user for mysqladmin ping
[SOGo] Use valid user for mysqladmin ping
2017-09-12 20:57:18 +02:00
andryyy
1ffed58956 [ACME] New version, better IPv4 detection 2017-09-11 21:51:17 +02:00
andryyy
ed4e8d301c Fix tabs 2017-09-08 18:41:02 +02:00
andryyy
b8c4093702 [Web] Allow a spam score up to 2000, fixes #556 2017-09-07 20:20:36 +02:00
Tobias
fcd0efc265 Change primary name of SSL certificate
Hostname as primary name for the SSL certificate, all other names will
be added as SAN
2017-09-03 19:41:47 +02:00
andryyy
1f90433429 [Dovecot, Postfix, SOGo] Fix redis log cleanup, fixes #542 2017-09-03 11:57:58 +02:00
andryyy
446907944a [Postfix] Fix missing ltrim 2017-09-01 23:40:23 +02:00
andryyy
008d500f4d [Compose, Dovecot] Fixed missing ltrim 2017-09-01 23:39:51 +02:00
andryyy
7351fcf1e3 [Compose, SOGo] New version, ltrim logs 2017-09-01 23:33:12 +02:00
andryyy
4d7bb26874 [Postfix] Add null rcpt for watchdog 2017-09-01 12:22:29 +02:00
andryyy
08d6b97ffa [Dovecot] New Dovecot and Pigeonhole versions; include delete2 option in imapsync 2017-08-31 10:36:10 +02:00
andryyy
4a0e3a433b [Fail2ban] Remove rule to detect disconnects without authentication 2017-08-30 22:27:33 +02:00
andryyy
b1213c51d7 [Rspamd] Dynamic ratelimit fixed, removed async redis request; Ready to implement per-user ratelimits via UI (tbd) 2017-08-30 21:42:39 +02:00
andryyy
7a2c0f2ee7 [Acme] Skip backup mx domains 2017-08-18 09:57:25 +02:00
André Peters
92eefc1288 Merge pull request #486 from mkuron/patch-1
Disable ClamAV phishing filter
2017-08-02 15:54:16 +02:00
Michael Kuron
dec64eef32 Disable ClamAV phishing filter
ClamAV has a phishing filter built in, but it generates too many false positives. We can disable it without risk because rspamd also checks for phishing itself.
2017-08-02 14:42:53 +02:00
andryyy
2eed7c05a1 [Dovecot] Push version, fix cronjob (fixes #310) 2017-08-01 14:02:00 +02:00
andryyy
c7484434dd [Dovecot] Remove master.pid if not running; Check for active imapsync besides lock file; Reconnect MySQL connection in imapsync_cron.pl 2017-07-31 08:19:02 +02:00
andryyy
33bf9f5c5d [Rspamd] Push image version 1.5, Rspamd 1.6.3 2017-07-26 23:07:01 +02:00
andryyy
8da02378b1 [Rspamd] Use Nginx IP in settings map 2017-07-26 23:04:49 +02:00
andryyy
03c614f749 [Postfix] Use Nginx IP in whitelist_forwardinghosts.sh 2017-07-26 23:04:27 +02:00
andryyy
b629089ff8 [SOGo] Rename reconf-domains to bootstrap-sogo and exec sogod after a successful bootstrap; Wait for SOGo to die and free listener 2017-07-24 23:25:04 +02:00
andryyy
5d5646df0c [Compose, SOGo] Update SOGo, give SOGo more time for a graceful restart, autorestart on any exit code 2017-07-24 22:53:23 +02:00
andryyy
83d485dd94 [Web, Postfix, Compose] Allow to add relayhosts per domain (+ plain and login authentication) 2017-07-22 20:39:54 +02:00
andryyy
ed33cb5f57 [Rspamd] ARC: Disallow login/domain mismatch 2017-07-21 11:03:35 +02:00
andryyy
5f5872f78b [Rspamd] Initial custom ratelimit support 2017-07-13 12:54:53 +02:00
andryyy
e6727b1fd6 [ACME] Iterate alias domains, use hostname in subject field 2017-07-13 12:51:52 +02:00
andryyy
9e92c4a2ad [Dovecot] Do not keep persistent logs in a container 2017-07-11 17:09:31 +02:00
andryyy
f8ae5158cb [Postfix] Do not keep persistent logs in a container 2017-07-11 17:09:20 +02:00
andryyy
a31819fd6c [SOGo] Log to a pipe to not keep logs in a container 2017-07-11 17:08:06 +02:00
andryyy
51660589d4 [Rspamd] Push version 1.3 2017-07-09 22:28:36 +02:00
andryyy
a324b1a385 [Rspamd] Fix DKIM siging (base64 hash line folding) 2017-07-06 15:57:33 +02:00
andryyy
69da02c144 [Dovecot] Important fix for Pigeonhole (downgrade) 2017-07-05 19:13:07 +02:00
andryyy
7f47af1d60 Fix destination hash 2017-07-05 12:31:52 +02:00
andryyy
87cc5f54ff [Postfix] Fix redis log destination hash 2017-07-05 12:03:12 +02:00
andryyy
2fadfee61a [SOGo] Remove thunderbird plugin generation, will move to docs 2017-07-05 10:22:48 +02:00
andryyy
de14d30e6b [ACME] Fix exit command (was using echo), fix for duplicate SAN (filter) 2017-07-04 21:32:58 +02:00
andryyy
dc463c3dda [Multiple] Push multiple logs to Redis channel for fail2ban-mailcow to read. Enables Fail2ban independently of used Docker logging driver. 2017-07-04 18:08:20 +02:00
andryyy
ed11e7586e [Clamd] Add SKIP_CLAMD variable to disable Clamd start 2017-07-04 18:05:44 +02:00
andryyy
ace247b3b9 [ACME] Unset name arrays before reusing them 2017-07-03 10:20:09 +02:00
andryyy
2cf9f71613 [ACME] Revert fix for empty additional_san, fix skip_ip_check in ACME 2017-07-02 20:18:22 +02:00
andryyy
cf902854d7 Merge, conflict fixed 2017-07-02 11:22:35 +02:00
andryyy
b7cb4ac9d5 [Fail2ban] Added more regex to match failed or disallowed logins to Dovecot, changed Mailcow to mailcow 2017-07-02 11:10:35 +02:00
andryyy
ae5ce6568d [Dovecot] Dovecot 2.2.31, Pigeonhole 0.4.19 2017-06-30 20:30:30 +02:00
andryyy
5ab11c0c1e [ACME] Show err instead of empty var when IP lookup failed 2017-06-30 20:29:55 +02:00
andryyy
d2048ccf20 [ACME] Be more verbose about IP address 2017-06-29 21:22:01 +02:00
andryyy
a6b60aebb8 [Fai2ban] Added auto-detection for container names; Allow multiple rules for each container; log rule id and container on match 2017-06-29 11:30:14 +02:00
andryyy
9040d456ed [acme-mailcow] Auto-detect container ids for restart; Restart containers after restore 2017-06-29 10:25:32 +02:00
andryyy
3d652dd3d0 Added more checks for acme-mailcow 2017-06-29 00:56:51 +02:00
andryyy
6d8438c01c - More checks for acme-mailcow (verify hashes)
- Autodiscover configuration file: Merge array from vars.local.inc.php
- Push acme-mailcow to 1.6
2017-06-28 23:22:51 +02:00