Deep Q forex

Deep Q Forex: Basic Discussion

This topic contains 15 replies, has 4 voices, and was last updated by  simplex 3 years, 8 months ago.

Viewing 16 posts - 1 through 16 (of 16 total)
  • Author
    Posts
  • #10916
    Moderator

      Latest discussion between Peter and me in the What’s New branch of this group has become too unclear, so I decided to start this forum topic inside the group – should be much easier to follow.

      I will start by reposting our latest 2 news posts for a start.

      s.

      A good trader is a realist who wants to grab a chunk from the body of a trend, leaving top- and bottom-fishing to people on an ego trip. (Dr. Alexander Elder)

      #10917
      Moderator

        simplex wrote:

        Interesting vid! The ‘Theory of Everything’, and physical Unified Field Theory being only a part of it ? Wow – this guy has really got a vision! And all this uncovered by AI algorithms. Forgive me if I’m a bit doubtful about this.
        Back to our ‘simple’ FX business. After having learnt a bit more about the general approach, I might see those 52% success rate in a different light now, and I wonder what would happen if we modified what the speaker called the ‘Environment’ in his slides:
        a) offer Deep Q a longer data history to learn from
        b) offer it a broader data universe, say all OHLC+V of all crosses of EUR, CHF, GBP, USD, CAD, AUD, NZD, JPY to learn from
        c) offer it a dynamic model of pair specific trading cost
        and then let it learn.
        Learn for how long?
        Which computing resources (CPU, RAM) provided?
        Question above all: What exactly is the ‘Goal’ we’re setting up for the learning process?
        Secondary question: is it wise to add additional time series, say indicator data, or would this compromise the uniqueness of the approach?
        Anyway, I’m still interested in some conceptual papers, rather than conference speaks. Sorry for being so oldfashioned!
        And I would be interested in some modeling details about that ‘marketcheck’ link you provided in your first post.
        s.

        A good trader is a realist who wants to grab a chunk from the body of a trend, leaving top- and bottom-fishing to people on an ego trip. (Dr. Alexander Elder)

        #10918
        Moderator

          Peter wrote:

          Interesting vid! The ‘Theory of Everything’, and physical Unified Field Theory being only a part of it ? Wow – this guy has really got a vision! And all this uncovered by AI algorithms. Forgive me if I’m a bit doubtful about this.

          Google brought his company for $400 Million, maybe the saw something of interest.

          Back to our ‘simple’ FX business. After having learnt a bit more about the general approach, I might see those 52% success rate in a different light now, and I wonder what would happen if we modified what the speaker called the ‘Environment’ in his slides:
          a) offer Deep Q a longer data history to learn from

          Yes you can feed the network more data anytime frame any market, I feed it with 66,000 rows of data, but you can add more if you wish

          b) offer it a broader data universe, say all OHLC+V of all crosses of EUR, CHF, GBP, USD, CAD, AUD, NZD, JPY to learn from

          Yes

          c) offer it a dynamic model of pair specific trading cost
          and then let it learn.

          Yes, this also fine and good idea

          Learn for how long?

          Data depend few minutes to a few hours

          Which computing resources (CPU, RAM) provided?

          The higher the better, large network models can from the use of super computer this can be hired on a time basis from Amazon,

          Question above all: What exactly is the ‘Goal’ we’re setting up for the learning process?

          Goal is simply a reward I choose maximum pips per trade an example you can use percentage correct as a Goal/Reward if you wish

          Secondary question: is it wise to add additional time series, say indicator data, or would this compromise the uniqueness of the approach?

          Yes this is fine and encouraged

          Anyway, I’m still interested in some conceptual papers, rather than conference speaks. Sorry for being so oldfashioned!

          Yes paper available at http://arxiv.org/pdf/1312.5602v1.pdf

          And I would be interested in some modeling details about that ‘marketcheck’ link you provided in your first post.

          Currently working on site will be available in the future

          (Above chat recompiled by simplex)

          A good trader is a realist who wants to grab a chunk from the body of a trend, leaving top- and bottom-fishing to people on an ego trip. (Dr. Alexander Elder)

          #10919
          Participant

            Interesting vid! The ‘Theory of Everything’, and physical Unified Field Theory being only a part of it ? Wow – this guy has really got a vision! And all this uncovered by AI algorithms. Forgive me if I’m a bit doubtful about this.

            <b><i><u>Google brought his company for $400 Million, maybe the saw something of interest.</u></i></b>


            Back to our ‘simple’ FX business. After having learnt a bit more about the general approach, I might see those 52% success rate in a different light now, and I wonder what would happen if we modified what the speaker called the ‘Environment’ in his slides:
            a) offer Deep Q a longer data history to learn from

            <b><i><u>Yes you can feed the network more data anytime frame any market, I feed it with 66,000 rows of data, but you can add more if you wish</u></i></b>


            b) offer it a broader data universe, say all OHLC+V of all crosses of EUR, CHF, GBP, USD, CAD, AUD, NZD, JPY to learn from

            <b><i>Yes </i></b>


            c) offer it a dynamic model of pair specific trading cost
            and then let it learn.

            <i><u>Yes, </u></i><i><u>this also fine and good idea</u></i>
            <!– [if !supportLineBreakNewLine]–>
            <!–[endif]–>

            Learn for how long?

            <i><u>Data depend few minutes to a few hours </u></i><i><u>
            <!– [if !supportLineBreakNewLine]–>
            <!–[endif]–>
            </u></i>

            Which computing resources (CPU, RAM) provided?

            <b><i><u>The higher the better, large network models can from the use of super computer this can be hired on a time basis from Amazon, </u></i></b><b><i><u>
            <!– [if !supportLineBreakNewLine]–>
            <!–[endif]–>
            </u></i></b>

            Question above all: What exactly is the ‘Goal’ we’re setting up for the learning process?

             

            <b style=”mso-bidi-font-weight: normal;”><i style=”mso-bidi-font-style: normal;”><u>Goal is simply a reward I choose maximum pips per trade an example you can use percentage correct as a Goal/Reward if you wish </u></i></b><b style=”mso-bidi-font-weight: normal;”><i style=”mso-bidi-font-style: normal;”><u><br style=”mso-special-character: line-break;” /> <!– [if !supportLineBreakNewLine]–><br style=”mso-special-character: line-break;” /> <!–[endif]–></u></i></b>

            #10920
            Moderator

              This morning I came across the following article: http://arstechnica.com/science/2015/02/ai-masters-49-atari-2600-games-without-instructions/

              The examples mentioned in many articles seem to be attempts to let a Deep Q network learn how to master Atari games without any instruction. I’m no expert in Atari games. At first sight an Atari game appeared to me much more complex than our ‘simple’ forex models.

              But there’s one big difference, IMO: a simple Atari game is a closed system, with a very limited number of different events displayed on screen for the user to deal with in order to increase his score. Main challenge for the user is to cope with new types of objects appearing on screen from level to level, and increasing velocity of objects.

              If we’re setting up a Forex model that is compact enough to be handled, this system is not closed. It is influenced by the ‘outer’ world not considered in our model, thus changing the preconditions of our modeling environment constantly.

              The real challenge in setting up such a model seems to be to design it in a way that model fluctuations triggered by external influences are limited in their impact regarding our model and its predictive power after the learning process.

              A good trader is a realist who wants to grab a chunk from the body of a trend, leaving top- and bottom-fishing to people on an ego trip. (Dr. Alexander Elder)

              #10925
              Participant

                This is interesting. Some kind of nnea neural network that can be used as a pr3dictor ?

                #10931
                Moderator

                  Just a quick note. I’ve read the following article this morning: http://www.financial-hacker.com/whites-reality-check/

                  One short quote:

                  It gets even more diffcult with machine learning algorithms that optimize weight factors and usually do not produce discarded variants.

                  Maybe it’s interesting to keep an eye on this ‘White’s Reality Check’ when considering to dig deeper in Deep Q.

                  s.

                  A good trader is a realist who wants to grab a chunk from the body of a trend, leaving top- and bottom-fishing to people on an ego trip. (Dr. Alexander Elder)

                  #10934
                  Participant

                    Hello Group

                    The bottom line is find an edge, in future I’ll add basic indicators such as moving average, bands or RSI as an input then monitor results

                    Also it possible to add close prices inter-market related data

                    The demo at https://youtu.be/QJHpaIZ7o_c  is based on 5 minutes raw price data nothing else , and it attempting to place 1)Buy Exit open, 2)Sell exit open  or 3)Do Nothing

                    If we have any  Javascript programmers in the group we can accelerate progress, source code will be posted at https://github.com/AIForex

                    Note anyone with CSV data can now cut and paste data window at ai.marketcheck.co.uk/Forex  and update group with results

                    Thanks

                     

                    Peter

                    ai.marketcheck.co.uk/Forex

                     

                     

                    #10935
                    Moderator

                      Hi Peter!

                      Could you possibly post some more details about the Deep Q environment and your personal experience with it?

                      • In case a group member would like to experiment for himself: which software and runtime environment is required?
                      • Where are download sources?
                      • Is it open source?
                      • Recommended tutorials or other resources?
                      • etc. …

                      You’ve done a first step: some Penguin members are interested to learn more.

                      IMO, you’re now suggesting a third or forth step before the second one is done. I would suggest to answer some basic conceptual questions now, and after that step back and consider what makes sense next. And most of all I would like to discuss basics, pro’s and con’s of this method. See my Atari vs. Forex post above: what’s your opinion about that?

                      To me, just adding some indicator values like MAs, Band, or RSI feels a bit like pure activism without a plan.

                      I would prefer to set up a plan first. But this is just my personal opinion, and you might have reasons to prefer a different strategy.

                      s.

                      A good trader is a realist who wants to grab a chunk from the body of a trend, leaving top- and bottom-fishing to people on an ego trip. (Dr. Alexander Elder)

                      #10939
                      Participant

                        Hello

                        If you would like further information on the source code and operation, theory  you can check http://cs.stanford.edu/people/karpathy/convnetjs/demo/rldemo.html

                        1)The first step has already been taken into add raw Forex data  to network

                        2) Add further inputs such as simply indicators

                        3) Build Node.js App  to integrate to metatrader 4

                        All source code plans and notes  would be available  at https://github.com/AIForex

                        The current results is interesting, and would expand with new findings and information

                        With regards to the Atari demo I see that the demo can be expanded to the Forex markets provided that the environment is adjusted to suit trading styles, the method, that I have incorporated is my style of trading …

                        I feel the Algo would benefit using good quality leading indicators and perhaps inter-market Forex data

                        I believe no Algo can would work on the markets without applying human trading experience,  the main point about this Deep Q it feedback Algorithm that is always learning and improving this is key feature as markets continually change

                         

                        Regards

                         

                        Peter

                        #10943
                        Moderator

                          Let’s consider I’d like to set up a local Deep Q environment.

                          What would be my steps to take before integrating to MT4 via Node.js?

                          Best,

                          s.

                          A good trader is a realist who wants to grab a chunk from the body of a trend, leaving top- and bottom-fishing to people on an ego trip. (Dr. Alexander Elder)

                          #10952
                          Participant

                            I have chosen to implement version shown in http://cs.stanford.edu/people/karpathy/convnetjs/demo/rldemo.html

                            Although  you research and write your own version by studying white paper at http://arxiv.org/pdf/1312.5602v1.pdf

                            The code is written in JavaScript so it can be ran in your browser, my suggestion if you want to run it locally then simply copy file from ai.marketcheck.co.uk/Forex  transfer to your  locally on your computer, I will make source code files aviable at https://github.com/AIForex

                            Hope this helps

                            Thanks

                             

                            Peter

                             

                             

                             

                             

                            #10956
                            Participant

                              Hi sim

                              I think you will like this one,

                              http://jmathfx.com/en/

                              sorry its Italy language,I still have many cant understand.

                              Regards.

                               

                               

                               

                               

                              #10962
                              Moderator

                                Thanks:

                                my suggestion if you want to run it locally then simply copy file from ai.marketcheck.co.uk/Forex transfer to your locally on your computer

                                And how would a typical process of system development and optimization look like working on your ai.marketcheck.co.uk/Forex platform?

                                Which is the format an algorithm / network is forwarded to MT after the learning process?

                                Which limitations do apply regarding number of rows / columns?

                                Re. optimization: how to avoid over-optimization / curve fitting? I think this would only make sense, if we implemented robustness against changing market sentiments as one ‘must have’ goal of the learning process. For example: the first part of the learning process is done on EURUSD exclusively, and then the resulting algorithm / network is tested on different pairs, defining the second part of the learning process, given the same timeframe and time interval. Do you think that this would be possible?

                                s.

                                A good trader is a realist who wants to grab a chunk from the body of a trend, leaving top- and bottom-fishing to people on an ego trip. (Dr. Alexander Elder)

                                #10963
                                Participant

                                  Hello

                                  The network Deep Q which is based on re-enforcement learning , the algorithm would continue learn on-line until results cannot be improved,

                                  The result of the network learning would be transferred  to MT,

                                  The idea using a Node.js app,  is to use data from MT (OHLC and indicator data ) to feed  Deep Q Nework as input

                                  The deep Q network would perform its calculation then the output can be feed back to MT for our use.

                                  This information can be displayed as an indicator or used as part of a trading system

                                  At the moment the demo uses 5 columns which is simply (Date Open High Low and Close) plans in  to expand columns so that indicator data can be used,  this can be intermarket close information or simple indicators such as moving average rsi, or bands

                                  The network can work with any Data EURUSD , GBPUSD, etc I have ran network using 66000 rows

                                  The you-tube demo  https://youtu.be/QJHpaIZ7o_c uses 5 min data,  The network goal is the maximise points/pips over time, for more robust network I can focus the goal on average profit per trade, of which is the single best measure  of network performance 

                                  My suggestion is to simply cut and paste your desired data set into the Deep Q Demo website ai.marketcheck.co.uk/Forex then monitor results

                                   

                                  Hope this helps

                                   

                                  Peter

                                  #10964
                                  Moderator

                                    Thanks:

                                    I think you will like this one, http://jmathfx.com/en/ sorry its Italy language

                                    I came across that a while ago. It seems interesting technically, but IMO, a test phase of 15 days on demo before having to open a live account is not enough to check it out. I certainly will try it when I’m having some more spare time for an intensive test.

                                    s.

                                    A good trader is a realist who wants to grab a chunk from the body of a trend, leaving top- and bottom-fishing to people on an ego trip. (Dr. Alexander Elder)

                                  Viewing 16 posts - 1 through 16 (of 16 total)

                                  You must be logged in to reply to this topic.