NodeJs Mysql connection example

[Participant]

Hi,
I tried to play with NodeJs to connect to Mysql yesterday, and it worked. Thanks to saver0 for the great information.
Make sure that we have followed the tutorials, download and install all needed software (xampp, nodejs, python, etc …) :
*) http://penguintraders.com/2014/09/24/setting-up-node-js-xamppmysql-for-mt4/
*) http://penguintraders.com/groups/fractal-arrow-prediction/forum/topic/installation-getting-started-guide/
*) https://github.com/PenguinTraders/FractalArrowPrediction

Make sure apache & mysql are started via xampp control panel, check from phpMyadmin too.
Make sure we have installed the module needed, ex: npm install async , npm install mysql, npm install underscore, etc…
And do not forget to export the path in system variable, see pedma’s post here: http://penguintraders.com/2014/09/24/setting-up-node-js-xamppmysql-for-mt4/

Create js script under C:\Node\ (ex: testing.js using your editor, i use Free Komodo Edit 8.5) like below :

var mysql      = require('mysql');
var connection = mysql.createConnection({
  host     : 'localhost',
  user     : 'xxxx',
  password : 'xxxxxxxx',
  database : 'fractals'
});

connection.connect(); 
var insertString  = {
  pattern: '0101000',
  barCount: 9,
  upOrDown: 1,
  count: 11,
  pos: 187,
  bars: '100111'
};

connection.query('INSERT INTO fractals VALUES ?', insertString, function (err, result) {  
  // ...
});
 
var queryString = 'SELECT * FROM fractals'; 
connection.query(queryString, function(err, rows, fields) {
    if (err) throw err; 
    for (var i in rows) {
        console.log('Pattern: ', rows.pattern);
        console.log('Barcount: ', rows.barCount);
        console.log('upOrDown: ', rows.upOrDown);
        console.log('Count: ', rows.count);
        console.log('pos: ', rows.pos);
        console.log('bars: ', rows.bars);
    }
}); 
connection.end();

Open command prompt, go to C:\Node , then run: node testing.js

If everything goes well, we can see some rows in fractals table of database fractals.
At this time, i still got some rows (repeated insert). Will check it again later.

Hope this help.
=smallcat

Attachments:

You must be logged in to view attached files.

[Participant]

Forget to tell that i have create a user & password, apply it to fractals database.
To do it, just open Xampp control panel, then click at “Shell” button (right, third from top – with black small icon).

Then type at # prompt :
# mysql -u root
(if you have set password for root user, then try: #mysql -u root -p )

to show database, type : show databases;
mysql> show databases;

i granted access to a new user “test” with password “testing2014” for example, by typing :
`
mysql> grant all privileges on fractals.* to test@localhost identified by ‘testing2014’;
mysql> flush privileges;
`

Note: it will automatically create user ‘test’ with password ‘testing2014’ & grant access to database “fractals”.

In this case, i must put at my testing.js file like this :

var mysql      = require('mysql');
var connection = mysql.createConnection({
  host     : 'localhost',
  user     : 'test',
  password : 'testing2014',
  database : 'fractals'
});

=smallcat

• This reply was modified 3 years, 7 months ago by  smallcat.
• This reply was modified 3 years, 7 months ago by  smallcat.

[Moderator]

This is Wonderful smallcat! You have done a great thing here. I’m so sorry that I didn’t see this thread. If you have any questions, please let me know.

Focus, Patience, Determination & Order in chaos

[Participant]

Thanks for giving the opportunity to join this group saver0. Honestly i just found some posts today (some of them are from last month or 2 months ago), they are very interesting. I did not find it some times ago …. :( . I will read all of them first, then try the connection to Mysql again. I will ask you later (i forget exactly what the error i got as i try to put data to mysql), thanks a lot saver0.

[Moderator]

Sure sounds good. I subscribed to the thread so I should get notifications now

Focus, Patience, Determination & Order in chaos

[Participant]

Hello. What are you trying to do by connecting nodejs with mysql? Is it related to some indicator?

[Moderator]

Node.js and MySQL is needed if/when a user wants to do more complex work easier. For example, the dashboard that I made, it runs in Node.js so I was able to make a web page that runs in Node.js that I can access from the browser and make a really awesome dashboard with all the features that I need. Something that I wouldn’t be able to do just in MT4. Also I run complex parallel executions in Node.js and running code directly in my GPU with cuda and I’m running things across two computers currently. Something that I can’t even imagine doing just with MT4

MT4 is good, its not bad. But if anybody wants to do more complex/creative stuff, then its best to code something up like in Node.js or create a .NET application or something like that. Also its not safe to run holy grail type code in MT4 because I heard that brokers can pretty much access your files through MT4. It’s a rumor, not confirmed but after seeing what happened, I wouldn’t trust MT4 at all anymore. It’s very easy for them to have a backdoor type access.

Focus, Patience, Determination & Order in chaos

[Participant]

lol I never imagined brokers accessing our MT4 folders … but its really possible …

So you export mt4 candles data to use within nodejs? If so, then you can use only naked data, right? Unless you recode the needed indicators…

ps: I’m developer too :) dont need to use non-thech terms with me :p
I’m asking just because I dont see nothing related to trading in smallcat’s posts … just a nodejs tutorial.

[Participant]

Just read this “What I have is a pattern classification algorithm that scanned the entire history of the 10 pairs to find high probability TZ patterns. Then a live pattern matching occurs to show me the highest probable levels as they develop.” in your first post in the other thread.

Is it Neural Network? Are you using some well known algorigthm or are you developing one?

Bro, do you have a super computer?? because to parse years of candle data would take days in my machine :P

[Participant]

Hi saver0, I do not understand the whole but got something after i read some of the posts here. It is interesting, and at the moment i am learning to code (i am a beginner Lol). i remember the old thread there by GG53 at FF, i think you all here want to do similar like that ? i mean, something like: storing historical data in database, call them later for calculation and analyzing, and then send the result to some where (as csv file for example). if i understand it correctly, the main programming language we will used here is Javascript instead of , and combine it with MQL / C# / Python / R / other programming language. This is nice, so we have general purpose idea : we can use what ever database we want (as long as it is supported by Apache web server), we can use what ever operating system we loved (windows, linux, etc), we can use what ever trading platform to trade (mt4, ninja, etc). Do i get it right?
Thank you for creating this nice website saver0 !!

 

 

• This reply was modified 3 years, 5 months ago by  Gio.
• This reply was modified 3 years, 5 months ago by  Gio.

[Moderator]

Feanor wrote:

lol I never imagined brokers accessing our MT4 folders … but its really possible … So you export mt4 candles data to use within nodejs? If so, then you can use only naked data, right? Unless you recode the needed indicators… ps: I’m developer too :) dont need to use non-thech terms with me :p I’m asking just because I dont see nothing related to trading in smallcat’s posts … just a nodejs tutorial.

I’m using Node.js to analyze the data and for DB purposes. I’m exporting candle data from MT4 and I also use MT4 to view charts as well and for oder placement. But everything can be done on Node.js end as well. One of my friends is actually developing a really complex and a killer trading platform using Node.js currently. It will blow everything out the water, including even the really expensive platforms such as Bloomberg and Reuters platforms. With Node.js I can actually connect directly to the broker via their API and get candle data and what not. I’m just using MT4 for now to speed up development and testing.

Feanor wrote:

Just read this “What I have is a pattern classification algorithm that scanned the entire history of the 10 pairs to find high probability TZ patterns. Then a live pattern matching occurs to show me the highest probable levels as they develop.” in your first post in the other thread. Is it Neural Network? Are you using some well known algorigthm or are you developing one? Bro, do you have a super computer?? because to parse years of candle data would take days in my machine :P

No, not a neural network. Below is my reasoning for not using neural networks that I posted on the Forex Factory thread. I’m not using a super computer but I am leveraging two computers and all 16 cores  

Also I just now started to experiment with CUDA in order to exploit the parallel computing power. GPUs these days are practically supercomputers

——————————————————————————

Actually I think the way I am doing is better than using neural networks or any other predictive methods because it’s not based on learned data or a summary. I have access to the entire 5million bar DB of each currency and all of its patterns. Think of it as a machine that never forgets the past (like our brain). All the data points are there and I can actually access it all (unlike our brain.. haha). And its the actual patterns from the past that I’m matching to as they show up live. In order for it to be a pattern it has to have occurred often enough in the past with a high probability of success. When there are such patterns, patterns repeat, hence why it is classified as a pattern. When the sample size is 50 million bars, it is safe to conclude that the patterns/results are statistically significant. The DB is continuously updated as new bars come. This I think is the most optimal way if it can be done (I have done it, so its possible).

If I take the neural network or machine learning type path to accomplish this, then I would actually have trouble trusting those results because I wouldn’t know why it gave the signal. With my method, I know exactly how the signal was generated.

Anyways, there are more than one way to skin a cat.. haha It’s all a matter of how you are going to use the data and to do what. Sometimes it’s not possible to remember each and every state and then be able to recall them, thats why we rely on neural networks and machine learning algorithms. It’s like think of this case, you have images of handwritten letters. Now what’s the best method to do handwriting recognition; A huge DB of all the possible letters written by man or training a neural network? Obviously it would be impossible to have a huge DB with all the handwritten words written by all the people in the world. But if we did have a DB with every word written by everybody every second, and if we could access that DB fast enough, then it would be possible to do a DB match for the pattern and we would get exact results. We place machine learning and neural networks in places where we cannot perform massive calculations. But if it can be done, then we would arrive at more accurate results.

So in Forex, my machine learning experience taught me that the only use for machine learning is when we have good sound logic that would work even without the use of machine learning. The machine learning comes into play when in need of summarizing or when going through multiple states or huge amounts of data. It’s nice for replacing long strings of if statements. But if the logic isn’t sound and if the logic cannot produce good results on its own, then machine learning will be as dumb as the logic, maybe even dumber. Because what happens is that through each neuron and in each level, you are increasing your error. If it’s to get a simple up or down type signal, then it could probably work, but for delicate levels that has to be exact like in my case, I think it would be impossible.

Focus, Patience, Determination & Order in chaos

[Moderator]

Gio wrote:

Hi saver0, I do not understand the whole but got something after i read some of the posts here. It is interesting, and at the moment i am learning to code (i am a beginner Lol). i remember the old thread there by GG53 at FF, i think you all here want to do similar like that ? i mean, something like: storing historical data in database, call them later for calculation and analyzing, and then send the result to some where (as csv file for example). if i understand it correctly, the main programming language we will used here is Javascript instead of , and combine it with MQL / C# / Python / R / other programming language. This is nice, so we have general purpose idea : we can use what ever database we want (as long as it is supported by Apache web server), we can use what ever operating system we loved (windows, linux, etc), we can use what ever trading platform to trade (mt4, ninja, etc). Do i get it right? Thank you for creating this nice website saver0 !!

 

You are absolutely correct about everything you said. That’s the true power these days. No need to be limited by MT4. We can program in any language that we like and leverage massive amounts of computation power available to us. We can even write apps to run directly in the cloud to utilize an entire grid of computers if any massive computational power, it will cost a bit but much cheaper than before.

Focus, Patience, Determination & Order in chaos

[Participant]

Good point about the neural networks … if you can access the whole past, why use a function to represent it 

 

So I imagine you have a good way to identify the patterns … could you tell your method of identifying patterns? 

I have a few ideas about that … I dont remember the algo names, I’ll need to search. Long time I worked with AI (really good times, I have to say  )

 

Manipuling raw data is really funnier than looking for setups on a chart 

I’m thinking in a lot ways here …  

 

Just give me a hint, please: do you limit the pattern to a limited maximum number of candles? Or your method can consider even a 100 candles pattern?

 

Thanks

[Moderator]

Yes, I’m planning to write more on this later. Just difficult to find the time with everything going on at the moment. I should be “finishing up” my development for the most part hopefully before this week ends and then I can do some more sharing 

My patterns are based on TZ theory. I just set the maximum and the minimum for H_Left and H_Right which I classify the patterns for. Are you pretty new to TZs?

Focus, Patience, Determination & Order in chaos

[Participant]

Saver0 wrote:

Yes, I’m planning to write more on this later. Just difficult to find the time with everything going on at the moment. I should be “finishing up” my development for the most part hopefully before this week ends and then I can do some more sharing My patterns are based on TZ theory. I just set the maximum and the minimum for H_Left and H_Right which I classify the patterns for. Are you pretty new to TZs?

 

I see … Yes, I dont know nothing about TZ (besides some early post on similarity thread and vlady’s thread).

I thought you would be identifying pattern just based on PA. This is the way I’ll be testing. I just created a group here to share my thoughts.

Just two more questions, please:

1) Which data source did you use for the candle data? I suppose you’re not just exporting from mt4

2) Do you store the patterns in database as well? Or keep them in memory?

[Moderator]

Feanor wrote:

I see … Yes, I dont know nothing about TZ (besides some early post on similarity thread and vlady’s thread). I thought you would be identifying pattern just based on PA. This is the way I’ll be testing. I just created a group here to share my thoughts. Just two more questions, please: 1) Which data source did you use for the candle data? I suppose you’re not just exporting from mt4 2) Do you store the patterns in database as well? Or keep them in memory?

Sorry for the delay in my reply. I wanted to take the time to reply to your questions and I was thinking about making some posts about TZs so that I can explain it quickly to anyone who is new to it. I’m still extremely busy so please give me some time to get to it. TZs are VERY extremely easy. There is nothing complicated about it. Just takes the time to put examples together and what not

You will see a thread or a blog entry from me on TZs. I’m thinking about creating a few different blog entries going from the very basic to more advanced stuff. But I need to find time for that. In the meantime, let me answer your questions

1) Which data source did you use for the candle data? I suppose you’re not just exporting from mt4

I’m using the data from Oanda and yes it is from MT4. I have a script that keeps my MySQL DB updated every 5 minutes. We can get candles directly from Oanda using their API as well http://developer.oanda.com/rest-live/introduction/  I plan to use this sometime soon.

 

2) Do you store the patterns in database as well? Or keep them in memory?

They are all stored in the DB and gets loaded into memory whenever needed for comparison.

Focus, Patience, Determination & Order in chaos

[Participant]

Saver0 wrote:

1) Which data source did you use for the candle data? I suppose you’re not just exporting from mt4 I’m using the data from Oanda and yes it is from MT4. I have a script that keeps my MySQL DB updated every 5 minutes. We can get candles directly from Oanda using their API as well http://developer.oanda.com/rest-live/introduction/ I plan to use this sometime soon. 2) Do you store the patterns in database as well? Or keep them in memory? They are all stored in the DB and gets loaded into memory whenever needed for comparison.

Hi Saver0,

First of all, thanks for your hard work and nice scripts. I have downloaded historical data from Oanda using your Node.Js script.  About updating your database in every 5 minutes, do you use the same JS script ?
And you said that you are using data from Oanda MT4, does it mean that for testing and developing we need to use Oanda MT4 (may be using demo account), instead of MT4 from other broker ?
Although we can put the downloaded historical data of Oanda to other broker’s MT4 platform…

[Moderator]

smallcat wrote:

Saver0 wrote:

1) Which data source did you use for the candle data? I suppose you’re not just exporting from mt4 I’m using the data from Oanda and yes it is from MT4. I have a script that keeps my MySQL DB updated every 5 minutes. We can get candles directly from Oanda using their API as well http://developer.oanda.com/rest-live/introduction/ I plan to use this sometime soon. 2) Do you store the patterns in database as well? Or keep them in memory? They are all stored in the DB and gets loaded into memory whenever needed for comparison.

Hi Saver0, First of all, thanks for your hard work and nice scripts. I have downloaded historical data from Oanda using your Node.Js script. About updating your database in every 5 minutes, do you use the same JS script ? And you said that you are using data from Oanda MT4, does it mean that for testing and developing we need to use Oanda MT4 (may be using demo account), instead of MT4 from other broker ? Although we can put the downloaded historical data of Oanda to other broker’s MT4 platform…

I’m doing all of the analysis in Node.js so I’m not exporting the data back into MT4. I’m using MT4 only for placing trades and to look at any other indicators. So it can be for any broker really.

As for keeping the candle DB updated, I added a bit more code. I didn’t share that part because I want you guys to try and figure it out. It’s pretty easy. What you need to do is use is

		setInterval(function(){
			function_that_gets_data("m5"); //Pass in the period
		}, 5*60*1000); //Telling the function to run every 5minutes

And then the function function_that_gets_data can be a function that calls the getData for each currency and the period that you pass in as a variable. Know what I mean?

Let me know if you get stuck

Focus, Patience, Determination & Order in chaos

[Participant]

Thanks a lot saver0, now all is clear … i understand now the reason you did like that  

[Author]

Posts