Network queries and performance

If you are using the main C++ distribution of wxWidgets, Feel free to ask any question related to wxWidgets development here. This means questions regarding to C++ and wxWidgets, not compile problems.
Post Reply
Natulux
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 242
Joined: Thu Aug 03, 2017 12:20 pm

Network queries and performance

Post by Natulux »

Hey guys,

here is a network performance question I want to put on you, because I do not have enough experience on this matter.
When searching the web for "network + performance + queries" or something simliar, they are all discussing SQL servers.
In my case though, I have an application on windows in a local network, using wxWidgets file objects, eg.:

Code: Select all

wxFileName fn ("\\\\172.16.x.x\\test\\exchange.txt");
if(fn.Exists())
{
	wxFileConfig *fc_file = new wxFileConfig(GetAppName(), wxEmptyString, fn.GetFullPath(), fn.GetFullPath(), wxCONFIG_USE_NO_ESCAPE_CHARACTERS);
	//...
}
And if the file exists, I am mostly opening it, reading it and maybe writing or copying something.

Or I am watching a directory on the server like this:

Code: Select all

wxArrayString as_files;
wxDir::GetAllFiles(ms_path.GetFullPath(), &as_files, "*.dat", wxDIR_FILES);
if (as_files.GetCount() > 0)
{
	for (int i = 0; i < as_files.GetCount(); i++)
	{
		//so something
	}
}
This works like a low grade share folder, where the server and each client have a directory to put things, which the other one needs.

Now, every computer in the local network is querying the server with these file operations with a timer and Im asking myself:
How much delay should the timer have?
If realtime is of no importance, I could use something like 10000ms, but what if I would like a quicker response, if the user is waiting for a return value? Like 500ms?
Testing this with the few machines in my local network is no problem, but I can't simulate a setting like one server for several 100 client PCs.

Do you have any experience, which delay for a greater network is still decent and doesn't shut down all other network communication?

[EDIT:] The setting, where my app is running, may differ from rather primitive networks with a handfull of PCs plugged into another PCs, which they call the "server" - to other settings, where there might be running a real (MS) Server and several hundered machines configured and controlled by rigid admins. ;-)

Thanks
Natu
User avatar
doublemax
Moderator
Moderator
Posts: 19160
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: Network queries and performance

Post by doublemax »

Are we talking about a local network over ethernet?

I don't have any practical experience with this, i would go for a limit of 10 accesses per second *on the server*. Based on that and the number of clients i would calculate the update rate.
Use the source, Luke!
ONEEYEMAN
Part Of The Furniture
Part Of The Furniture
Posts: 7477
Joined: Sat Apr 16, 2005 7:22 am
Location: USA, Ukraine

Re: Network queries and performance

Post by ONEEYEMAN »

Hi,
Did you try the Curl library?

Thank you.
Natulux
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 242
Joined: Thu Aug 03, 2017 12:20 pm

Re: Network queries and performance

Post by Natulux »

Hey guys
doublemax wrote:Are we talking about a local network over ethernet?
Yes, ethernet local network it is, indeed.
doublemax wrote:I don't have any practical experience with this, i would go for a limit of 10 accesses per second *on the server*. Based on that and the number of clients i would calculate the update rate.
You mean 10 accesses per second in total? So if I had 100 clients, each of them could access once every 10 sec. Thats quite a delay.
Do I understand correctly, that each FileExists() query would be one access? Even if they come one after another? I don't think, that there is any smart concatenation of queries, but sometimes there is more to it than I believed. ;-)
ONEEYEMAN wrote:Hi,
Did you try the Curl library?
I have used curl (executeable via command line) before and it is great for eg. HTML querying, but why exactly would you suggest to use curl here? I could achieve exactly what I wanted with wxWidgets so far, Im just concerned to shut down the whole network by setting my timers to small.

Cheers
Natu
User avatar
doublemax
Moderator
Moderator
Posts: 19160
Joined: Fri Apr 21, 2006 8:03 pm
Location: $FCE2

Re: Network queries and performance

Post by doublemax »

You mean 10 accesses per second in total? So if I had 100 clients, each of them could access once every 10 sec.
Yes. It is a quite conservative number, you could try 100, maybe even more. Without a real-word test this might be hard to estimate. But the problem is that the accesses are not distributed evenly over time, theoretically they could all come in at the same time.

As i don't know the "bigger picture": Are you sure there is no better way than polling these files frequently?
Use the source, Luke!
Natulux
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 242
Joined: Thu Aug 03, 2017 12:20 pm

Re: Network queries and performance

Post by Natulux »

doublemax wrote:Without a real-word test this might be hard to estimate. But the problem is that the accesses are not distributed evenly over time, theoretically they could all come in at the same time.

As i don't know the "bigger picture": Are you sure there is no better way than polling these files frequently?
Well, maybe I just need to take my chances then. I guess there are better options. Maybe consistenly query a socket server or similar.

My main goal atm (though not the only reason for my question here) is to deploy updates of my software, which might only be some flags remotly written in a config file or rather some new files.
eg.: My App has a server App which it is connected to. When the server address changes (when it is moved), my apps loose their connection. I want to be able to remotly correct those paths. For that and other purposes, they have a watchfolder, which they monitor for command files and/or new files.

But well, that is another topic actually ;-)

Cheers
Natu
Nunki
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 235
Joined: Fri Sep 14, 2012 8:26 am
Location: Kontich, Belgium
Contact:

Re: Network queries and performance

Post by Nunki »

Hi Natu,
Another approach could be that when a file is stored on the server in a folder, let the application do this and in addition maintain some sql table with key data in it like file-name, file-location, search keys etc.. Instead of polling the server to read all stored files, just do a query on the sql database table to see if you need to read a file. If so, only read this file or when the data you need is managable in size, store it also in the database. So you won't be needing the actual files at all.
Sometimes it is good to take a different approach than the first one that pops into mind.

best of luck
Nunki
Natulux
Filthy Rich wx Solver
Filthy Rich wx Solver
Posts: 242
Joined: Thu Aug 03, 2017 12:20 pm

Re: Network queries and performance

Post by Natulux »

Hey Nunki,

thanks for your suggestion. This could be a good addition, if I run into performance problems. :-)

Cheers
Natu
Post Reply