Polish users cut off from TalkChess

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Guenther
Posts: 4045
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: Polish users cut off from TalkChess

Post by Guenther » Sun Dec 06, 2020 12:15 pm

PK wrote:
Sun Dec 06, 2020 8:27 am
@Guenther, you have a nice crawler over there! I have one friend who can investigate options of putting at least programming part of the forum somewhere, so stay in touch.
Haha, I did not assume someone would have linked an external 158GB file to the forum...
http://talkchess.com/forum3/viewtopic.p ... se#p806561
(added a max size for non htm files now @30000000 Bytes)

User avatar
Guenther
Posts: 4045
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: Polish users cut off from TalkChess

Post by Guenther » Tue Dec 15, 2020 8:56 am

Guenther wrote:
Sun Dec 06, 2020 12:15 pm
PK wrote:
Sun Dec 06, 2020 8:27 am
@Guenther, you have a nice crawler over there! I have one friend who can investigate options of putting at least programming part of the forum somewhere, so stay in touch.
Haha, I did not assume someone would have linked an external 158GB file to the forum...
http://talkchess.com/forum3/viewtopic.p ... se#p806561
(added a max size for non htm files now @30000000 Bytes)
Well, after crawling for around nine days I had to abandon this for now. It blocked my (old) computer for too long and the result somehow was not convincing enough. Moreover after reaching around 85% it needed another day to add 2-3 percent, with still growing link numbers.
(Noticed also that it would not be possible to synchronize with already available links e.g. in CPW, because of a randomly added hex number with 4 digits, in the middle of the new created links)

Probably I should have found better settings at the beginning.
May be I will try it again later, with more cautious settings and less depth.
OTH it would be much simpler, if ChessUSA would create a database backup and make it available somewhere.

Pi4Chess
Posts: 253
Joined: Mon Nov 16, 2020 11:13 am
Full name: Manuel Rivera

Re: Polish users cut off from TalkChess

Post by Pi4Chess » Fri Dec 18, 2020 4:32 pm

Guenther wrote:
Sat Dec 05, 2020 5:27 pm

I am currently in the process of saving the most important parts of the forum by an old tool, which still seems to work like a charm.
(I don't want to put too much stress on the host server so I limited the bandwidth for downloading)

This will take a while and I have to check first, if the saved archives work at all later.
If all works out as desired, I will report back in a few days.
I don't think I will save everything and will filter manually 'not so useful' content, at least from the General forum.

Image

I have no clue how big it will be and hopefully it won't crash my computer over night ;-)
Oh, I forgot to say I started of course with the Programmers forum :)
After this I will try the General forum. The others are not that important.

Edit:
A first check is very promising - everything is working as expected in the saved archives structure!

BTW ofc I also tried wayback on this and in fact talkchess was saved hundredths of times in the past, BUT the saved archives
never go deeper than the thread/post titles, which means those are useless.

First guess for the programmers part ~5GB (total links will still increase further)
This will make the General part something like 20-50GB
It reminds me of old Website aspirator HTTrack ^^ It was not easy to configure parameters but you could exclude link to files more than x Mb i remember.

User avatar
Sam Hull
Posts: 5804
Joined: Tue Mar 14, 2006 8:19 am
Location: The Cherokee Nation
Full name: Sam Hull

Re: Polish users cut off from TalkChess

Post by Sam Hull » Wed Dec 23, 2020 4:57 pm

PK wrote:
Sat Dec 05, 2020 7:35 am
peter wrote:
Sat Dec 05, 2020 6:15 am
But even less I would simply let the data stored here go without even trying to save them in one way or the other.
I agree. But Talkchess is no longer a place where you can reliably store and retrieve information.
Email Quentin Turner at YMC&G directly with your problems and concerns ([email protected]). If there is a sufficient volume of requests I would expect there to be some action. I have forwarded all requests sent to me about unblocking, but the last couple of months I have gotten no replies.

-Sam-

User avatar
mclane
Posts: 18533
Joined: Thu Mar 09, 2006 5:40 pm
Location: US of Europe, germany
Full name: Thorsten Czub
Contact:

Re: Polish users cut off from TalkChess

Post by mclane » Wed Dec 23, 2020 11:41 pm

Sam you are not the solution. If there would be a solution to the problem you would not be informed.
What seems like a fairy tale today may be reality tomorrow.
Here we have a fairy tale of the day after tomorrow....

User avatar
hgm
Posts: 26539
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Polish users cut off from TalkChess

Post by hgm » Sun Jan 24, 2021 4:55 pm

The following idea occurred to me:

Wouldn't it be possible to set up a 'relay server' at a non-blocked IP address, which does nothing other than forward the URL request to TalkChess, fetch the data, and then forward that to the original requester? People could use that as if it was the original TalkChess website. (Although it would be a little slower.) I suppose that for every page it relays it would have to substitute any occurrence of the string 'talkchess.com' to its own domain name, so that people who follow the links would again go through the relay server.

User avatar
mclane
Posts: 18533
Joined: Thu Mar 09, 2006 5:40 pm
Location: US of Europe, germany
Full name: Thorsten Czub
Contact:

Re: Polish users cut off from TalkChess

Post by mclane » Sun Jan 24, 2021 10:22 pm

Good idea HGM
What seems like a fairy tale today may be reality tomorrow.
Here we have a fairy tale of the day after tomorrow....

Krzysztof Grzelak
Posts: 1092
Joined: Tue Jul 15, 2014 10:47 am

Re: Polish users cut off from TalkChess

Post by Krzysztof Grzelak » Tue Apr 13, 2021 4:50 pm

After all, gentlemen, it will be done or not.

Sopel
Posts: 121
Joined: Tue Oct 08, 2019 9:39 pm
Full name: Tomasz Sobczyk

Re: Polish users cut off from TalkChess

Post by Sopel » Sun Jul 25, 2021 9:56 am

The technology is finally there. Amazing what things are possible in 2021.

User avatar
Rebel
Posts: 6346
Joined: Thu Aug 18, 2011 10:04 am

Re: Polish users cut off from TalkChess

Post by Rebel » Sun Jul 25, 2021 6:23 pm

Hardly get access since yesterday, the notorious Forbidden You don't have permission to access /forum3/search.php on this server once again. Since today no access at all. Need to use a VPN connection.

Is Quentin Turner blocking IP's again?
90% of coding is debugging, the other 10% is writing bugs.

Post Reply