8 January 2014 – Mail Exchange Issue

Maintenance to improve stability.

Update 13 – 21/01 17:03
We schedule a maintenance, tomorrow morning at 05:00, to improve the performance and stability of our mail servers. During the maintenance all mail service will be temporary unavailable. No mail will be lost.

Maintenance will start on 22/01 around 05:00 and will take about 30 minutes.

Update 12 – 20/01 10:30
After the works of this weekend it seems that we have found the origin of the slow responsiveness of the mail servers. By replacing the defect disk controller in our main storage server the speed of our mail servers increased drastically. Monday morning we had our first real stress test by you as customer (Monday morning is the most busy time frame in the week for our mail servers) and the response time of our mail service stayed normal. We keep on monitoring our mail system but we presume this issue finally is solved.

Update 11 – 18/01 10:29
Yesterday evening we found another problem with the storage systems, we completely replaced one of the storage systems, this device is now replicating  data, and should be finished today. We also needed to replace a defective disk controller in another storage system, this night this controller started working as it should again. Due to the fact that the load is lower in the weekend and at night, everything should anyway work optimal. We need to wait until Monday to see if the problem rises again.  We are very sorry for the lack of communication yesterday, we were working very hard, but had no updates.

Update 10 – 17/01 10:57

The overnight monitoring didn’t give the expected result. Therefore our technical team will perform another maintenance at 11h. We keep you up to date as soon as we have more feedback. For your information: meanwhile webmail is back online. Please clear your browser cache if you still arrive on the issues page when connecting to webmail.

Update 9 – 16/01 16:40
Together with our software supplier we are currently implementing a new possible solution. During the next hours and night we will monitor this new implementation to see if it gives the necessary speed gain. We keep you up to date as soon as we have more feedback.

Update 8 – 16/01 10:52
Meanwhile our technical team contacted our hardware supplier in order to find the possible cause and solution.  We keep you up to date as soon as we have more feedback.

Update 7 – 15/01 20:15
Our technical team had to bring webmail off line in order to continue to search for a solution for our e-mail delays. We keep you up to date as soon as we have more feedback.

Update 6 – 15/01 17:04
Our technical team is working non-stop to solve the mail issue. There are several problems causing e-mail to slow down. Our technicians are excluding several possible issues one by one. Every time we get one step closer to a solution. As soon as we have more feedback we will keep you informed.

Update 5 – 15/01 11:03
The scheduled maintenance of this night did not completely resolve the issues, therefore we need to schedule another maintenance as soon as possible. Our mail systems will go completely offline at 12:00 for 20 to 25 minutes.

Update 4 – 14/01 14:39
The data has been rebuilt but our systems are still causing time-outs. Our network team will perform a maintenance tonight (Starting at 01:00) to resolve the issues. During this maintenance you will not be able to receive any e-mails. We expect to resume full service by tomorrow morning.

We can confirm that no data is or will be lost.

Update 3 – 13/01 13:13
We expect that the delay for receiving email messages will be solved within 48 Hrs. The delay is caused also due to the high load during working hours. Edpnet is doing everything to speed up ongoing processes 

Update 2 – 09/01 10:30
Our data storage system experienced a big failure. No data is lost and everything kept on running. The system is currently rebuilding a lot of data and responds slow. You can experience time-out when retrieving your e-mail. Please try again after a few minutes if you have a time-out. Completely resolving this issue will take some days.

Update 1 – 14:43
The issues were temporary resolved but have returned, our network team is still looking in these issues. We thank you for your patience.

Due to a failure on our e-mail system we have to perform an unforeseen maintenance right now. You won’t lose any mails. Please be patient if your e-mail isn’t working at the moment.

Thank you for your understanding.
 
Edpnet apologizes for any inconveniences.


09 Januari 2014

Vanwege het falen van onze opslagsystemen worden mails momenteel vertraagd verzonden. Er is geen data verloren gegaan en de systemen draaien nog steeds.De vertraging wordt veroorzaakt door het heropbouwen van onze systemen. Het is mogelijk dat u ook tijdelijk een foutmelding krijgt, gelieve het dan enkele minuten later nogmaals te proberen.

Het volledig oplossen van deze storing kan tot enkele dagen in beslag nemen.

Update 9 – 20/01 16:00
Na werken van dit weekend lijkt het erop dat we de oorzaak van de vertraagde mails hebben gevonden. Door het vervangen van een defecte controller op onze hoofd opslagserver is de snelheid van het systeem drastisch verhoogd. Maandagochtend was de eerste stresstest van ons systeem door u (de klanten) aangezien een maandagochtend  één van onze drukste dagen is voor onze mailserver. De responstijden blijven normaal. We blijven alles monitoren.

Update 8 – 18/01 10:29
Gisteravond hebben we nog een probleem gevonden met een opslag systeem, we hebben dit volledig vervangen, en dit toestel is ondertussen data aan het repliceren, dit zou vandaag afgerond moeten zijn. We hebben ook een defecte disk controller dienen te vervangen in een ander opslagsysteem, deze nacht is deze nieuwe controller opnieuw optimaal beginnen werken. Omdat in het weekend en ‘s nachts de load sowieso lager is, zou alles nu optimaal moeten werken. We dienen tot maandag te wachten of het probleem terug zou opduiken. Onze excuses voor de beperkte communicatie gisteren, we waren hard aan het werk, maar hadden geen updates.

Update 7 – 17/01 11:16
De monitoring van de laatste implementatie heeft afgelopen nacht bevestigd dat we nog niet het gewenste resultaat hebben. Daarom wordt nu, om 11u, nog een volgend onderhoud uitgevoerd. Van zodra wij hierover feedback hebben, wordt u op de hoogte gebracht. Intussen is webmail terug beschikbaar. Mocht u nog op de storingen-pagina terecht komen, gelieve dan even uw browser cache te verwijderen.

Update 6 – 16/01 16:40
In overleg met onze hardwareleverancier werd een mogelijke oplossing uitgewerkt.  Deze wordt op dit moment geïmplementeerd.  In de loop van de avond en de nacht wordt verder opgevolgd om te zien of dit de nodige snelheidswinst geeft zodat er geen vertragingen meer optreden.

Update 5 – 16/01 10:59
Intussen heeft ons technisch team contact met de hardware leverancier en zoeken ze samen naar de mogelijke oorzaak van het probleem.   Wij houden u verder op de hoogte van zodra er feedback is.

Update 4 – 15/01 17:02
Ons tech team is nog steeds volop bezig, het is niet een welbepaald duidelijk probleem dat de vertraging in de mails veroorzaakt. Ze zoeken stap voor stap uit door te testen en de resultaten van de ingrepen te controleren. Telkens komen we een stap dichter bij de oplossing. Wij informeren u van zodra we meer nieuws hebben.

We willen nog even bevestigen dat er geen data is of zal verloren gaan.

Update 3 – 15/01 11:22
Het onderhoud van deze nacht heeft de problemen nog niet volledig verholpen, daarom dienen wij nogmaals een onderhoud in te plannen. Deze zal om 12:00 starten en er wordt geschat dat er gedurende een half uur geen mails zullen worden uitgestuurd.

Update 2 – 14/01 14:42
Onze systemen zijn momenteel terug opgebouwd maar er zijn nog blijvende problemen met het vertraagd ontvangen van berichten. Daarom plant ons netwerkteam deze nacht om 1 uur een onderhoud in. Gedurende dit onderhoud is het niet mogelijk om berichten te ontvangen. We verwachten dat alle systemen terug normaal draaien tegen de ochtend.

Update 1 – 13/01 13:22We verwachten dat de vertraging tijdens het ontvangen van berichten zal worden opgelost binnen de 48 uren. De vertraging wordt mede veroorzaakt door een te hoge belasting tijdens de werkuren. Edpnet doet er alles aan om dit proces te versnellen.

Edpnet excuseert zich voor het ongemak.


6 September 2013

Er zijn deze ochtend problemen geweest in verband met inkomende gesprekken voor onze EDPnet Telephony klanten. Deze gesprekken werden rechtstreeks naar de voicemail doorgestuurd. Uitgaande gesprekken bleven werkzaam.

Volgens ons netwerkteam werd het probleem opgelost.
Onze excuses voor het ongemak.
 

There have been issues this morning in regards to receiving calls via EDPnet Telephony. Incoming calls were directly forwarded to voicemail. Outgoing calls were still working.

According to our network team, this issue should now be resolved.
We appologize for the inconvenience.


19 June 2013

There is currently an outage on the authentication service of edpnet. It is impacting all of the internet connections. If you are offline your modem can’t establish an internet connection anymore. Our engineers are working on a solution. We expect the issue to be solved during the evening.

We want to ask you to restart your modem from time to time. Your internet connection will re-establish itself automaticly or after a reboot as soon as the problem is solved.

Update 19/06/2013 18:15:
The problem is now solved.

Our apologies for the inconvenience.


Toegang tot bepaalde sites

Beste klant,

Momenteel is toegang tot bepaalde sites moeilijk tot niet mogelijk. Edpnet is zich hiervan bewust en we zijn traffiek aan het herrouteren. We trachten dit binnen het komende uur opgelost te krijgen.
Onze excuus voor het ongemak.

Some sites have bad reachability or not reachable at the moment.This is known problem at EDPnet , we are trying to reroute traffic. We suspect it will be done within a hour.
Our apologies for any inconvenience

Update:
Problem solved.


29 April 2013

There is currently an outage in East- and West-Flanders impacting internet and telephony services.
Edpnet is trying to solve this as fast as possible.

Update 29/04/2013 10:21: There’s an outage in our supplier’s network. We’ll let you know as soon as any ETR is available. Sorry for the inconveniences.

Update 29/04/2013 11:15: The fault is detected, splicers are already working on it. The connection should be restored in a few hours. We’ll keep you informed of any new information.

Update 29/04/2013 12:33: We’ve just received a confirmed ETR. The issue should be solved by 14h00.

Update 29/04/2013 13:13: It has been found out that the issue is more complicated than expected, there’s a large section of fiber to be replaced. Splicers are currently working on it. We expect it to be solved within 2 hours. Thanks a lot for your patience and understanding.

Update 29/04/2013 15:56: First of all, we’re sorry for the delay in the update. The splicing has been just finished, all customers should get online again. The end of the works is not confirmed yet though, we’ll keep you informed.

Update 29/04/2013 16:29: Our supplier confirmed that all the affected fibers were replaced. All services are up and running with no further issues since 15h55.
We apologize for the inconveniences it caused and thank you for your patience.

Edpnet apologizes for any inconveniences caused.


20 February 2013

You might experience slow connections due to a fiber cut in our core network. All traffic is rerouted. Because we can’t use full capacity you can experience slow connections to several international (especially German and Dutch ) locations.

Update 20/02/2013 17:15 : Our NOC engineers tweaked the routes. Currently the congestion on the network is normalized. The fiber cut is not solved yet. You can still experience slow connections to several international (especially German and Dutch ) locations. 

Update 21/02/2013 07:30During the night the fiber cut has been resolved. All traffic is routed back via the original routes.


We apologize for the inconveniences.

22 January 2013

You might experience slow connections.
There seems to be a DDOS attack towards our systems, we are looking into it.

Update 0:10
The problem should be solved, we will look into avoiding this for the future.

We apologize for the inconveniences.


22 oktober 2012

Er zijn momenteel problemen met het verzenden van e-mails via onze relay omwille van een probleem met een spamlijst.

Edpnet tracht dit zo snel mogelijk op te lossen.

Update (23/10/2012):

Dear customers,

We have a huge mail issue since yesterday. For the second time spamhaus.org listed our non-spamming IP ranges. You are unable to send mails to people who use spamhaus.org as a filter to receive e-mail.

Up to one month ago we always had a good contact with spamhaus.org. They informed us (on IP basis) where we had our spammers/abusers and we took contact with them to solve the issue.
Since one month spamhaus.org decided to take extreme measurements towards edpnet. Blocking entire IP ranges for several times because of spammers in our network who we deactivated as soon as we were aware of it.

As an ISP we do not do any filtering of traffic and we do allow fixed IP users to have their own mail servers. It seems that this policy of unfiltered & open internet is no longer possible when such measurements are taken. It seems that we as edpnet will have to redefine our policy regarding traffic filtering to prevent such issues as we have now.

Until now we only reacted on complaints and we did not pro-actively scan all traffic for possible abuse.

We do believe that over the years we already took quite a lot of measurements to get the internet free from abuse and spam as we closed port 25 for all our dynamic IP users and we implemented a state of the art Cisco Iron Port mail system.
Nevertheless we still do not do any traffic filtering and we don’t want to.

Currently we have a huge issue and we do not get any response from spamhaus.org.
We do encourage all our customers to contact spamhaus.org.

You can help us by :

Ask your de-listing on : http://www.spamhaus.org/lookup/
Re-tweet our message using the button below:

Tweet to @spamhaus

Edpnet verontschuldigt zich voor dit ongemak.

Update (23/10/2012):

Het probleem lijkt opgelost, spamhaus heeft edpnet van de lijst verwijderd. 


22 October 2012

There are currently problems sending e-mails via our relay due to spam listing.

Edpnet is trying to solve this as fast as possible.

Update (23/10/2012):

Dear customers,

We have a huge mail issue since yesterday. For the second time spamhaus.org listed our non-spamming IP ranges. You are unable to send mails to people who use spamhaus.org as a filter to receive e-mail.

Up to one month ago we always had a good contact with spamhaus.org. They informed us (on IP basis) where we had our spammers/abusers and we took contact with them to solve the issue.
Since one month spamhaus.org decided to take extreme measurements towards edpnet. Blocking entire IP ranges for several times because of spammers in our network who we deactivated as soon as we were aware of it.

As an ISP we do not do any filtering of traffic and we do allow fixed IP users to have their own mail servers. It seems that this policy of unfiltered & open internet is no longer possible when such measurements are taken. It seems that we as edpnet will have to redefine our policy regarding traffic filtering to prevent such issues as we have now.

Until now we only reacted on complaints and we did not pro-actively scan all traffic for possible abuse.

We do believe that over the years we already took quite a lot of measurements to get the internet free from abuse and spam as we closed port 25 for all our dynamic IP users and we implemented a state of the art Cisco Iron Port mail system.
Nevertheless we still do not do any traffic filtering and we don’t want to.

Currently we have a huge issue and we do not get any response from spamhaus.org.
We do encourage all our customers to contact spamhaus.org.

You can help us by :

Ask your de-listing on : http://www.spamhaus.org/lookup/
Re-tweet our message using the button below:

Tweet to @spamhaus

Edpnet apologizes for any inconveniences.

Update (23/10/2012):

The problem should be solved, spamhaus removed us from their list.