Linux DStar Development

New Project Name

2010-10-13T06:39:00.000-07:00

Please note that due to existing trademarks and the potential for infringement, the project name has been changed to:

StarGate

NI-Star v2 has arrived

2010-09-17T09:17:00.001-07:00

NI-Star v2 is released, supporting multiple (up to 3) Node Adapters.

Support has been tested with:

- Satoshi v6.20
- Satoshi v7.08
- Dutch*Star v0.1.28-2

A full installation package for the v1 code will be available via the UKIT support team.

Modifications to run v2 are available on request.

73 David

Announcing the Imminent Arrival of NI-Star

2010-06-29T14:13:00.000-07:00

Non-Icom Smart Technologies for Amateur Radio

The release of NI-Star is getting closer. The system now functions with Satoshi v6.17/6.20, v7.x and Fred's latest code.

Significant changes have been made to the code to make it easier to package and release, and this process has taken much longer than anticipated.

Please bear with me....

David

Why so quiet ?

2010-04-27T09:03:00.000-07:00

Simple. Eyjafjallajokull.

I was supposed to be in Bahrain for one week, then Florida for two.... It ended up as Florida for three weeks, 3 days.

In the meantime, I've been working on the user-facing web-pages with considerable help from others. They are now complete and running on GB7MH.

Mostly because I can't think of anywhere else to say this, I am going to say it here.

Thank You ! to the citizens of Florida for turning what could have been a week of expense and boredom into an enjoyable experience. You were simply GREAT.

Busch came up with FREE tickets for SeaWorld and Aquatica
Disney came up with FREE Rover tickets for the parks
The Zoo gave us 50% off
Even the Miniature Golf courses reduced rates !
Hotel dropped prices, restaurants too.

Everyone kept saying how sorry they were for us.

In short, we were treated in our adversity in the manner which I would hope I treat people in theirs. It counts for a lot, and we are VERY GRATEFUL.

Dayton 2010

2010-02-26T15:43:00.000-08:00

For those interested, I will be at Dayton 2010, and will have my repeater live on-air in the show.

Hopefully, I'll be able to get a G2 and D-Plus link out of the hall as well.

I'm presenting on the software (time to be advised), and will be at a booth for some of the time - alternatively, you'll be able to call me on the repeater !

73 David - G4ULF

Release Plans

2010-02-26T15:39:00.000-08:00

Hi All

I've had many, many requests for code, and here are my plans.

At the moment, the installation is manual and time consuming. My plan is to take a break from development as the code seems substantially complete and concentrate on automating the installation via scripts.
As part of this, I want to add a user addition/deletion web page - the current system of flat text files is not working. I have an offer of help from Colin at GB7NE for this (thank you).
The code will then be made available via the UKIT.
I can then concentrate on new developments/bug fixes.

Please bear with me whilst I package it all up....

v6 Test System on-line

2010-02-07T12:48:00.000-08:00

At last, I have a stable v6 system up on G2 and D-Plus.

I've got two v6 boards build - one using v6.12 which works fine, and one using v6.16 which is not so stable. Now, I tried to flash the v6.16 unit with the v6.12 code, and immediately hit the code protection, so I need to ask Satoshi very nicely to provide me with the correct load.

I've also built a new system on CentOS 5 on a Mini ITX with 512MB RAM which using an EPIA-M 10000 and an 8GB SSD. This is MUCH faster and better than the unit installed at GB7MH, so at some point, I will be swapping this out for a v6 unit.

David

Logging changes - future support

2010-02-02T03:22:00.000-08:00

I've added a tag to /var/log/dsipsvd.log that indicates the version of software to D-Star Monitor. This in turn is picked-up by the dstarusers.org page so that machines running my software can be identified.

Now, future support. As I move into new "challenges" at work and the number of installed machines increases, I need to work out how I'm going to allow any group that wants to to build/install a system without compromising the software and without my having to run remote installation. Not quite sure how to work this one out yet, so stay tuned....

David

G2_poll bug

2010-01-21T01:08:00.000-08:00

I've found a bug in the g2_poll process that maintains contact with the Trust Server.

The issue is with new station registration and the interaction with D-Plus.

Presently, when a station is added to the G2 database, I insert into the sync_rip table. D-Plus checks the sync_mng table. This is not an issue for RF stations - the first time they transmit, the mng entry is added. However, for DV-Dongle users that will never update over the air, this means that the MNG entry is never created, so the callsign is not added to the users.txt file used by D-Plus.

g2_poll changed to send initial sync_mng entry after new user insertion.

RadCom Article on GB7MH

2010-01-10T19:50:00.000-08:00

Look out for the Feburary edition.

David

Latest Changes

2010-01-10T18:57:00.000-08:00

I've been trying to track down the source of occasional node adapter crashes for sometime, so I've added some new routines to ensure that only one thing is going on at a time. Version 0.988a adds some new routines:

1) At the end of each local transmission, it checks that the buffer is empty and that the PTT has been dropped. This is a good indication that the cycle is over.

2) I've added some 10ms "usleeps" between the calls that set-up the outgoing header.

So far, so good....

2010 - Happy New Year

2010-01-04T02:42:00.000-08:00

Happy New Year to all Blog followers !

And now an update.

I have a rare TX bug which appears after a local repeat - I think it is a timing issue between ending the transmission and sending the G2 status information. At the moment, I wait for the TX buffer to show that it's empty (0xFF) before I send any more commands to the Node Adapter. I then wait 500ms before setting the D-Star headers and PTT on to send the "RPT ?" or similar out.

But, sometime, the Node Adapter crashes after the TX buffer - my suspicion is that in those instances, 500ms isn't enough delay, because there are still packets in the send buffer of the Node Adapter. So rather than look for an empty buffer, I've modified the code to look for PTT OFF - i.e. all frames and the tail have been sent.

Let's see where this goes....

David

Bug/Feature and v6 update

2009-12-09T02:34:00.001-08:00

As you all know, when you add a user to the G2 database, the local gateway selects an IP address that is unique for that base callsign and sends that to the Trust Server.

So, I have had some issues with adding users due to duplicate IP addresses, but this has been really exercising me as in my database the IP address was clear. What I've found is that deleting users from G2 is a two-step process - the user is flagged for deletion and then at some point in the future when all gateways are synced, the user is purged.

Big problem - in the current trust network, the gateways are never really in-sync. So where I'd been pulling callsigns out and re-using IP addresses immediately, I now flag them with a deleted flag instead.

v6 tests progress very well. The code looks very stable - running on Ubuntu 9.10 on an EEE 901 PC and a pair of Tait radio modemes quite happily.

I've been able to change the USB routines to deal in 12-bytes as high-speed USB supports longer data frames, and this means that I can now link the outgoing buffer to the incoming buffer much more accurately. The net effect is that I have been able to reduce the time-lag through the repeater and nearly eliminate the transmit overhang on local repeat.

David

v6 Beta

2009-12-02T15:01:00.000-08:00

So I received my v6 Beta PIC from Satoshi (thank you !) two days ago, plugged it in and it worked as expected from day one.

I've now modified my code (releater 0.985) so that it recognises whether it is attached to a v4 or a v5/6 board - the main reason for this is that on v5 and v6 I use S/N rather than COS. I've also included a new value in the .cfg file that allows you to set the S/N trigger level.

The next stage will be to investigate having two v6 boards connected to the same gateway.

Bugs....

2009-11-30T10:47:00.001-08:00

I've found a CPU Hog in g2_poll. Resolved.....

Progress with the v2.06 board !

2009-11-27T10:35:00.000-08:00

Well, at last I've had some time to look at my v2.06 board issues and I've resolved them.

There are some really nice features on the v5 code, but mostly the inbuilt S/N replacing the COS. Setting the S/N at "100" in software and replacing the COS mechanism is great.

Also, the v2.06 board opens up the possibility of using Satoshi's latest v6 code which is in Beta. There are two superb features that I need in the v6 code:

1) Inbuilt bootloader - no more pulling the PIC out and re-programming - it can be re-programmed over the USB interface !

2) The ability to change the product ID - that means that I can have several boards connected to one gateway machine, in other-words a multi-port repeater (in the future...)

As soon as I get my v6 Beta PIC from Satoshi, I'm going to start playing.

73 David

S/N Board Progress

2009-11-15T00:52:00.000-08:00

Well, nearly two weeks ago I tested the S/N board, but since then I've not had a chance to install the mod on site.

And this week looks even busier....

To COS or not to COS - that is the question...

2009-11-03T13:43:00.000-08:00

I took GB7MH off-air most of yesterday and today as I tested some new code designed to help with reception of weak GMSK signals. Whilst it was not a total success, I have learnt quite a bit from the testing which I’ve now been able to replicate at home on the test system.

The problem I’m trying to solve is that reception of GMSK should be able to go to significantly noisier signals than it’s FM equivalent. However, at the moment, GB7MH is running on an FM receiver and uses the squelch to detect a “valid” carrier. This makes no sense for GMSK – I should just check for a readable GMSK waveform and decode. But at the moment, I’m gated by the FM squelch.

The effect is that rather than descend into higher bit-error rates for weak signals, the reception seems to cut out from perfect to nothing. The answer is to leave the squelch wide-open and just detect the GMSK modulation on it’s own merit. Then progressively weaker signals should just degrade to a pre-defined S/N point, where I cut-out.

So, I wound the squelch wide open – and caused all sorts of problems, because noise, especially the kind we had with lots of rain yesterday “looks” like GMSK. The effect was long-lived reception of rubbish. I tried several schemes to work around this (counting valid re-syncs in the GMSK, looking for header/end-of-transmission markers), but none really worked, and a sudden spike of noise kicked off a local transmission using the callsign of the last valid station heard. All very confusing.

The solution lies in not using COS, but using a S/N value from the GMSK demodulator. But this isn’t built into the v1 Satoshi board, and I just can't get the v2 board to play nicely.

So, I'm going to build the G4SZM "S/N to COR" circuit and use this to drive the COS pin. Then I can wind the squelch wide-open and just rely on true GMSK traffic.

That's the theory......

David

Update - 31/10/09

2009-10-31T14:38:00.000-07:00

GB7MH and GB7BM have been performing very well for the last few weeks.

Minor changes to the code in the last couple of days around the database synchronisation:

1) Bug in the "New User" code - when a user is added to the G2 sync_rip database, the Node ID sent is only 7 bytes long - I was sending 8.

2) Change to the way incoming updates are handled. I now run a "SELECT" using the "last_mod_time" before an "UPDATE" in Postgres. This significantly cuts down the number of changes on each update and really speeds the process up.

GB7MH is still surviving with an 800MHz Via EPIA, 4GB CF card and little else. Truly a D-Star micro-repeater.

73 David

And the next bug is...

2009-10-12T14:50:00.000-07:00

.... not exciting the threads neatly.

I've modified their operation so that any failure is trapped and an immediate pthread_exit is called. This has been working fine for 4 days.

G2 Update/Routing Issues Fixed

2009-10-07T16:53:00.000-07:00

Hit some problems today when testing G2 routing and updates "in anger." I was getting issues where the two threads were locking, and causing a seg fault in the code.

The reason appears to be that originally I had the two processes within the main code, rather than running them as separate threads, and I had declared global values which were now used outside of the main body. If one thread access them at the same time as the other - disaster.

So far, so good, but I need to keep a close eye on this one.

GB7MH D-Star Improvements

2009-10-04T09:50:00.000-07:00

Some improvements to the software over the last few days:

1) The way that the re-sync bits are generated has been changed. The entire frame is reformed to ensure that the re-sync pattern (0x55, 0x2d, 0x16) is sent every 21 frames, regardless of the incoming stream. This has eliminated mid-over beeps during G2 and D-Plus sourced QSOs. The same logic has been applied to local repeat which assists marginal incoming transmissions, and to outgoing G2/D-Plus packets.

2) The detection of a valid station on the input has been changed so that only valid D-Star headers will cause the repeater to open. FM/Squelch/Invalid-CRC headers will not open the local repeat functionality and are effectively ignored.

3) Lost sync on Internet-sourced packets. This is a difficult one to crack, but I can now detect and compensate for up to 2 seconds of lost DV frames, in-filling with silence.

73 David - G4ULF

GB7BM QRV

2009-10-01T02:36:00.000-07:00

GB7BM covering Birmingham City Centre went live with the Linux software yesterday.

GB7MH has been providing excellent service for the last 7+ days with minimal issues.

Bugs, bugs, bugs

2009-09-16T23:46:00.000-07:00

What a weekend. So, the box went live on Thursday evening, and by Friday it was obvious there were problems.

In no particular order, the problems I found and fixed:

1) Local repeat. The biggest pain. Satoshi's firmware requires a constant supply of packets to transmit - if they are lost, then PTT is switched off. Now, say you are a mobile user or a marginal signal - then DV frames will be lost. I was simply reading in frames off-air, and echoing straight back out - a BIG mistake. As soon as I lost a frame, PTT went off. The solution - a dual-leaky buffer system. Collect incoming frames and pre-fill the buffer on Satoshi's board. When the buffer reports it is full, then switch the PTT on and keep topping up the queue. So, leaky bucket #2 is in the Satoshi board, leaky bucket #1 in my code. I also watch for the spacing between incoming frames - if I see lost frames then I in-fill with Null Voice frames. This was a major re-write of the code.

2) FM lock-up. I was able to stall the repeater by forcing the squelch to open - it sat and waited for a DV Header which never happened. So I've implemented a timer and COS check.

3) Lock up on RPT1/RPT2 set. Still work-in-progress. Essentially, anyone that had RPT1 set incorrectly (or not set at all) was also causing the repeater to lock-up. This was because I had changed the DV Header handling as above and the result was coming back as a valid Header, but not enough data. I've worked around it for the time being - what should happen (I think) is that the repeater should allow you to work locally, but at the end of your transmission should respond with "RPT GB7MH B" that auto-fills in some rigs. I'm going to test this later on a real Icom system to see what responses come back.

4) Tail echo. Because of the way that I pre-fill the playout buffer, then raise PTT, when the transmission finishes, there is audio left in the buffer. I have two choices - 1, cut the transmission off early or 2, playout the remaining buffer to the end. I do 2 at the moment, but this leads to an overhang of about 1.5 seconds. In other words, you hear the end of your own transmission after you de-key. Now, in some ways, this is good - you can be sure that you've hit the repeater, but in other ways it is bad - I hate hearing my own voice. The solution is to pre-fill the playout buffer quicker, but I'm using low-speed USB, so I'm limited to 8-bits per transaction which takes around 1.7ms. Once I've sorted out the user responses (UR ?, RPT ? etc) then I'll think about other solutions.

GB7MH QRV

2009-09-11T14:44:00.000-07:00

At long last, after many months of work by the members of the Repeater Group, GB7MH is on-air.

That's the good news. The bad news is that as expected, a live repeater is a very different beast to a test repeater, and today was plagued by a nasty bug associated with marginal signals locking the repeater. I've modified the RX routine.

I also found an issue with the G2 registration code - the packet sent to the Trust Server was malformed, but this has now been rectified. Two new D-Star users were registered today on the K5TIT network.

I'm sure that's not the end of the bugs - I'll be watching the box like a hawk for the next few days.

August - slow month

2009-08-11T15:06:00.000-07:00

Quick update. Nothing much happens in August....

The software is working well, with one or two minor bugs resolved over the last few days.

The hardware for GB7MH is now complete - however, we have a lack of Internet connectivity at the site which is being worked on. I'm playing around with a 3G router - results are OK, but the G2 database update comes in at 1.4GB/month, and that is going to cost £££.

Early September and we should be on-the-air.

Fully Operational

2009-08-02T11:03:00.001-07:00

Postgres has now fully replaced SQLite3 on my modules:

g2_poll
g2_receive
dstar_g2

Now I need to re-write the dplus-readdb module which is part of Robin Cutshaw's D-Plus program to populate the users.txt and gateways.txt.

GB7MH has been working faultlessly on G2 since Friday evening, syncing roughly every 50 minutes to the Live Trust Server. REF001C has been playing in the background most of the weekend.

Never use Global Variables.....

2009-07-31T12:39:00.000-07:00

So I found the issue with the Postgres calls. Nothing to do with Postgres; everything to do with my lack of programming skills.

For whatever reason, I had quite a lot of globally-declared variables in my program, one of which was "char poll[3] = {0x00 0x30 0x26}"

Now, it just so happened that "int poll" is declared through the libpq-fe.h header used for the PQlib interface to Postgres. Result ? Segmentation Fault.

Memo to self: next time, use namespaces.

Grrr - Postgres

2009-07-30T08:20:00.001-07:00

Well, it had to happen. The g2_poll and g2_receive modules work well with Postgres, but dstar_g2, the main call handling/routing code resolutely failed to run with the dreaded "Segmentation Fault" on the stock Postrges with CentOS4.

So, I've upgraded to Postgres 8.4, but...... the database structure has changed, and I didn't dump it before upgrade.

Now I need to delete the database, re-create and then re-populate.

What a pain..................

SQLite is no more

2009-07-29T03:44:00.000-07:00

It looks like SQLite really can't hack it. I've moved the two G2 synchronisation processes g2_receive and g2_poll over to Postgres, and the improvement is staggering.

A database sync which was taking anything up to 2 hours to complete now finishes in less than 7 minutes !

There is SQL logic in dstar_g2, the main program, which needs modifying to use Postgres rather than SQLite, and that will take a few more days, but that should be all.

David

Live QSO - G4ULF to G3SMU

2009-07-23T07:21:00.001-07:00

Last night at about 22:00 BST, I put a call out to G7LWT from GB7MH.

G3SMU answered - operated by G7LWT ! We had a 10+ minute QSO using G2 routing between GB7MH and GB7NM, the Zone Repeater with G3SMU and G7LWT.

I then relinked to REF005A and had conversations with a couple of stations on the reflector.

There are a couple of G2 database sync issues - the SQL update is taking a huge amount of time, but I think I know why that is. There is also a TCP Window Scaling problem on the G2 SQL side, but again, I'm pretty sure that I know what that is.

None of this is service impacting other than the G2 updates can take a little longer to replicate than is ideal.

All-in-all, I think I can admit to being pretty pleased with progress !

Linux G2 Announcement

2009-07-21T15:22:00.001-07:00

At the end of last week, GB7MH, currently awaiting installation at our repeater site, was connected to the live Icom G2 Trust Network.

The repeater uses my Linux-based G2 compatible code, Robin Cutshaw's DPlus code (unmodified) and Pete Loveall's D-Star Monitor code (unmodified).

The repeater is a modified Tait T800 operating in full GMSK mode with 6.25kHz of bandwidth.

The interface board is the Satoshi first-generation GMSK Node Adapater.

The unit is capable of full callsign routing to/from the live G2 network including station updates. G2 registration is available via a self-service webpage, much the same as on Icom G2 nodes.

The system is now on test pending site installation in the next few weeks and usually connected to REF005A. Installation could be delayed if any service-impacting bugs are found - particularly with the G2 SQL replication.

I would like to thank (in no particular order) the following people who have made this possible:

- Satoshi Yasuda
- Jim McClellan
- Pete Loveall
- Robin Cutshaw
- Darren Storer
- Iain Philipps
- The members of the Ashdown Forest Repeater Group
- Anyone else that I asked a dumb question of and was kind enough to help me.

NOTE: Until I am happy that the system is stable, software will NOT be available. The G2 network is very fragile, and I need to be 100% sure that this system works without impacting other users before I'm prepared to release it.

73 David - G4ULF

G2 Station Registration

2009-07-11T15:10:00.001-07:00

Well, it looks like I've cracked the G2 IP address registration. I now randomise an IP address in a /30 subnet, check to see if it's a duplicate, and if not allocate it to the starting station address.

Then I insert into the G2 database. Whilst I don't have a neat front-end yet (I have to manually enter the new callsign in a text file) and I don't have the ability to add other suffixes in the allocated range (yet), I can now add stations into the G2 Trust server from my software, which means that I do all that a G2 system does.

I've also been testing and debugging the callsign updates to the database.

Now going to work on a nice front-end for the G2 registration and move GB7MH to the live G2 network as soon as possible.

Tait T800 and GMSK

2009-07-08T07:57:00.000-07:00

Put the finishing touches to the Tait T800 repeater today - it really is superb for GMSK.

I had to modify the VCO to instructions from G8VIQ and G3VVT, and I still need to align the receiver (or more correctly, get someone who knows what they are doing to align the receiver), but there is a step-change difference in bit-errors over a non-dual-point modulated TX.

Audio is crisp and clear, every RX callsign appears, and the text messages are error-free.

G2 Tests Update

2009-07-03T16:22:00.000-07:00

I now have my two repeaters, GB7MH and G6TBA connected to the G2 Test Trust Server very kindly provided by Jim McClellan.

I've been able to move stations between the two repeaters and make direct G2 routed calls between two stations.

All-in-all, pretty happy with things.....

Update - end of June 2009

2009-06-27T13:16:00.000-07:00

The modified G2 code continues to work very well with the latest release of Satoshi's firmware, 4.48.

One more change to the operation of my code - I have decided to remove my emulation of D-Plus from the code and use Robin Cutshaw's original D-Plus images.

These have been tested and work - the major benefit now is that my repeater will look and feel just like any other Icom G2 repeater, having D-Star Monitor and D-Plus, and any other application that can talk to the G2 RPC/Gateway packets.

I'm now in the process of modifying a Tait T800 repeater which will go into service as GB7MH. This is already reporting it's position and "Last Heard" information to www.dstarusers.org. It is also registered to the G2 Test Trust server kindly built by Jim McClellan.

Also registered to the same Trust server is a second repeater, G6TBA B. This exists only on dummy load, and it's purpose is to provide an on-going test-bed once GB7MH is moved to the live Trust server.

D-Star Monitor

2009-06-17T14:29:00.000-07:00

Finished the emulation of the RPC packets this evening - they are bound to the lo interface, so they never actually leave the box.

Then installed the D-Star Monitor application from AE5PL. With a couple of tweaks (various libraries missing, my total lack of understanding of what I was trying to achieve), I now have the Linux repeater reporting it's position via the D-PRS module in D-Star Monitor, and registering itself and LastHeard information to www.d-starusers.org !

Now need to test fro robustness.

How cool is this ???

2009-06-16T02:24:00.000-07:00

Sitting in a hotel with limited access to my development system at home, so I've built a Portable D-Star repeater:

2 x Tait Radio Modems
1 x Satoshi Board
1 x Laptop running Ubuntu 8.04

I've had to VPN using PPTP to my home router to pick up a real IP address, but it works !

D-Star Monitor

2009-06-09T14:15:00.000-07:00

OK - I've started the work to integrate D-Star Monitor to the repeater.

So far, I've added the regular 60-second poll - this sends "GB7MH S" to a loopback address to indicate that it is alive and well. Hopefully, DSM will be able to bind to the lo0 interface to be able to pull this out.

Next step is to re-create the DV stream on port 20000 and the "Location" annoucement using the same socket.

Other work - still trying to track down a couple of corner cases on audio, but all-in-all, working very well.

Update - June 2009

2009-06-05T09:22:00.000-07:00

It's been a very long time since I updated the blog, so here goes.

1) The repeater is working well. I've been listening to various reflectors and cross-connecting to the K5MIJ system over G2. The G2 database sync has stayed up for nearly 3 weeks with no issues to-date.

2) GB7MH, which will be the first repeater running this software, has finally been approved, and I am awaiting the licence.

3) I am now building a second Linux repeater for dev-test purposes.

4) With considerable assistance from the good folks at the K5TIT team, I am attempting to install DStarMonitor on my repeater. This will add all the usual reporting and monitoring features associated with Icom G2 nodes. I take my hat off to their tolerance with my dumb questions.

More news as it happens.

G2 Integration Testing

2009-05-12T04:51:00.000-07:00

Quick update as work is taking almost all of my time at the moment....

With the amazing assistance of Jim, N5MIJ, the Linux repeater is now connected to a lab-based G2 Trust Server which has an Icom G2 70cm repeater connected.

Database synchronisation works - the MNG, RIP and GIP tables are pulled down correctly and updates sent as new devices are added.

There is one issue that new stations that have an entry in the RIP table aren't updating the Trust Server when they are heard on the Linux system. This is due to me forgetting to migrate the "Check RIP" routine from flat text-files to SQLite3, and is probably due to me working far too late into the evening. This has been corrected and will be tested as-and-when time permits.

The next step will be to attempt a G2-routed QSO through the two repeaters - probably not until next week at the earliest.

Meanwhile, the Linux G2 system continues to perform very well as an RF-to-DPlus gateway.

73 David - G4ULF

G2 Migrated to SQLite

2009-05-05T09:41:00.000-07:00

I've completed the migration from flat-text files to SQLite3 for the G2 database and tested.

On my test system, this all works fine - stations that have been registered to the G2 Trust System but not yet been heard by a repeater will appear in the sync_rip table, and then update to the sync_mng as they are heard. Likewise, stations that have already been advertised through the sync_mng table will update the G2 system to their new zone rp.

Now time for testing on a "real" G2 system....

Progress Report

2009-05-01T11:13:00.001-07:00

I've been running the 4.18 code continuously for 3 days, and I have to say, results are superb.

Audio quality is as near perfect as D-Star ever gets both on recieve and transmit, and I've been able to iron out a couple of logic issues now that I'm less concerned about getting stable voice.

I've had to re-think the G2 routing, and to that ends, I've re-written the G2 code using a light-weight SQL database, SQLite

There are two problems:

1) I am NOT an SQL person, but learning quickly....
2) There was a bug in SQLite 3.6.12 that caused a seg fault on an INSERT with a timestamp. Solution was to upgrade to 3.6.13

Just running the last few debugs, but the SQL database now takes the place of the former flat-text file lookups in both the G2 synchronisation code and the main repeater code.

All ready for G2 integration testing over the next couple of week.s

4.17 At Last; and a few other code changes

2009-04-27T14:48:00.000-07:00

For quite some time, I've been worried about having to have an accurate timer on the repeater to pace the packets out to the Node Adapter. Whilst I was able to work a system where I had reasonably accurate pacing, it still was subject to glitches as the OS went off to do something else.

So, I've been playing with getting the Node Adapter to pace the packets - i.e just throw the frames at the board over the USB and let the return code tell you when the buffer is full.

Now, this is all fine, and is the same as Mark McGregor (KB8KHM) has been doing with his software for Windows - what was really bugging me was that I just couldn't get this to work at all for Linux.

But, I've tracked down the problem.

The 3.xx code was written for USB low-speed - I could get this to pace correctly, but I wanted to use the 4.xx code (due to many, many bug fixes, especially on RF receive). The 4.code just didn't want to play ball.

Until Satoshi released a low-speed version of 4.17 - plugged it in, fired up the code, and - VOILA ! Perfect audio....

So, it looks like there is an issue with the pacing on the USB 1.1 Full-Speed mode, or on libusb 0.1....

This is Very Good News, and allows me to move forward with other developments such as the G2 integration testing.

Satoshi 4.xx Code

2009-04-19T01:31:00.000-07:00

Still playing with the 4.xx code. I'm starting to wonder if there is a basic incompatability between the USB on the PIC and the libusb stack in CentOS 4.x.

Now, that would be a huge problem. The board I have is VIA EPIA 800 - a 586-compatible fanless unit. CentOS 5.x refuses to load on it - although the board reports 686 compatability, it just isn't compatible enought for CentOS 5.

The 3.26 code seems to perform very well for me, but others state that it is not up to scratch as a repeater.

I'll start researching other distros that may support the Eden with a newer kernel and libusb.

"Send Messge" bugs

2009-04-19T01:25:00.000-07:00

A few bugs cleared-up today:

1) Send Ident - now sends repeater callsign in Icom format (e.g. GB7QQ/RPTR) and date/time as message

2) Other text messages (linking/unlinking/status responses) weren't working reliably due to turning the PTT off too quickly. I now allow at least 1 full frame to be sent.

RX Audio Improvements

2009-04-13T14:15:00.001-07:00

Brain-wave this evening.

Rather than searching for 0x55 0x2d 0x16 in the incoming stream and using that to sync the playin buffer, I decided to try using the packet sequence number and stream ID.

So, each time I receive a 0x00 on the stream I'm listening to, I forward the counter.

Result ? A huge improvement in the ability to sync to the incomign stream.

Phew.

Satoshi 4.06 Code

2009-04-13T11:31:00.000-07:00

I rebuilt using the Satoshi 4.06 code today rather than the 3.26 I've been using.

Big mistake.

There seem to be several bugs with calling "AD_GET_STATUS" (to find the status of the carrier and end-of-transmission frame) which send the adapter into a tail spin. Also, there is a problem sending the playout frames - I send at a regular 20ms which 3.26 finds very warm-and-cozy, but 4.06 objects to violently, sending me all sorts of -ve numbers and causing my software to down-speed the playout.

This in turn means garbled voice on receive, so I'm backon 3.26.

I'm using CentOS 4.7 as my host, and the USB implementation is 0.1.8 which I know has issues, but I seem to be able to work around those with 3.26 - not a bit of it with 4.06.

I also decided to try Mark MacGregor's mechanism of a 21-byte ring buffer for playin, and a sequential lagging buffer for playout, but instead of pacing to the Satoshi board, just throwing the packets at the card until they were accepted.

Another Big Mistake. This might work on Windows, but Linux hates it, throwing all sorts of USB errors in dmesg. I even managed to panic the system....

Audio Problem Investigations...

2009-04-04T14:33:00.000-07:00

Well, that was VERY hard work. I've been looking into the issues with broken voice, and it appears that it has everything to do with the sync frame.

In short, the sequence 0x55 0x2d 0x16 has to be repeated every 21 frames, or else things break down. What I had been doing was assuming that taking the frames off "the wire" would be sufficient - it wasn't.

So, now I split the slow-speed data from the audio (why, oh why did they put a data channel in the audio bit-stream in UDP ?????) and rebuild the sync frame by sending every 21 frames. I then in-fill periods of lost sync with silence packets - this has the effect of breaks in the audio if the pattern is lost.

Things sound pretty good on RX now...

Audio Problems - again

2009-04-02T08:52:00.001-07:00

I've been spending my time cleaning things up, and I'm still not happy with the audio quality under lossy conditions.

Everything is OK provided the sync frame appears from the far end nice and regularly, but if I loose the sync over a couple of frame periods, things start to break, and the audio takes several sync frames to recover.

I have to ensure that I keep the playout pointer behind the playin buffer as well - this is really stretching my brain...

Progress on G2

2009-04-01T00:52:00.000-07:00

No update on this site for quite some time, but I've been working away in the background on the next phase of my project, and I have made good progress.

I can now emulate an Icom G2 Node for databse synchronisation, accept incoming DV sessions from other G2 nodes and route calls by callsign using the data in the database sync.

Whilst I believe that this is all working properly, I don't have access to the global G2 trust server, so I cannot route to real G2 repeaters. I've made initial contact with the K5TIT team who are looking at the infrastructure needed for testing and development.

The next stage of the project is to try to understand the registration process that Icom have used. They appear to be using internal IP addresses based on blocks of 8 - exactly how they allocate those addresses is a complete mystery at the moment, but I'm sure there is a pattern somewhere.

Bug Fixes

2009-03-10T04:22:00.000-07:00

Well, I've been quiet here for a while, because I've been "soak-testing" the software.

All-in-all, thing seem to be working well on the Mini ITX system. I've upgraded to the latest kernel and just checked memory/CPU usage, and things look good.

I have two outstanding bugs on the general software:

1) Under certain circumstances, login to dplus repeats every 2 seconds. I suspect that this is related to the logic that checks to see if the connection has been lost, and then attempts to re-authenticate, but I can't find the trigger that sets it off. Under investigation.

2) On local repeat mode, a noisy station into the receiver where the last frame isn't caught will leave the repeater hanging in transmit. I need to add logic to check for COS as well as a valid DV Header.

QSL VA3UV on XRF005 B

2009-02-27T15:15:00.000-08:00

Just had a QSO with VA3UV via XRF005 B on the Mini ITX system.

He was on Asterisk to his RPTDIR node using a SIP phone.

So, that's:

IC-91 - Satoshi board - Linux Node ---- INTERNET ---- RTPDIR - SIP

Who says Amateur Radio isn't cutting edge ?

Bug Fix and testing progress

2009-02-27T15:11:00.000-08:00

1) Found an issue in the End of Transmission packet sent during a dextra over - essentially, it wasn't sent. Fixed.

Testing progress:

I've built a Mini ITX system, 512MB RAM, EPIA VIA 5000 unit. It won't run CentOS 5, but it does a passable job of CentOS 4.7.

So, I've built using a 4GB CF card as the hard-drive, which means, as the 5000 is fanless, I have no moving parts at all.

Software loaded and compiled (OK, I had to upgrade the libusb to 0.1.12-5), but tests out perfectly. The box is less powerful that the thing I was using before, so X-Windows has gone, but as a standalone repeater, it's perfect.

Bug Fix/Major New Feature

2009-02-25T11:19:00.000-08:00

Bug Fix

1) The issue with crashing on no matching node being found is fixed.

New Feature

D-Extra node support. In the same way as users are able to link to D-Plus, I've added support for linking to the D-Extra network.

Bug Fixes/New Features

2009-02-24T06:17:00.000-08:00

Bug Fixes

1) GPS issue when GPRMC sent. The speed/direction indicators weren't formatting correctly when the direction was in triple digits. Resolved

2) There was an issue with the Node ID being read as 9 characters rather than 8 from the config file. This was breaking the callsign display. Resolved

3) If the "link-to" callsign is not found, the program crashes with a segmentation fault - on-going

New Feature

1) The ability to set TX Invert and RX Invert from the config file.

Testing Outcome

2009-02-18T14:37:00.000-08:00

Well, things are really moving now. I tested on REF013C with help from Darren G7LWT and Declan, M0TMX this evening, and results were good.

Both receive and transmit audio to/from the reflectors and more importantly to the distant linked repeaters was good, with negligable packet loss.

Onto the next challenge - getting this to run on a Mini ITX, fanless machine.

Transmit re-work

2009-02-16T09:09:00.000-08:00

Well, since running the tests on REF013C last week (thanks to Darren, G7LWT), I've been trying to chase down a couple of bugs.

1) During the first 300ms of local reception, there is an issue where audio played to the dplus network suffers from a break of up to 130ms. This was due to buffering on the Satoshi board, and meant that I had to rework the pacing.

2) Also on local reception, there was an issue whereby the first few frames sent to dplus were showing as excessively errored. Again, this proved to be to do with holding in the software in order to get packets to the Satoshi board.

3) Version 3.37 of the Satoshi firmware incorporates transmit and receive inversion in software - this is very useful for interfacing to some radios, but unfortunately, I found it to break my audio and timing. I'm now back to 3.26.

Changes made:

- New outbound pacing logic to the board. This puts the received audio off-air into a large buffer, and waits until the board is able to accept the inbound frames. However, it does not slow down outbound dplus packets which was what was causing the pacing issues.

- Change to the transmit logic - it appears that dplus is very fussy when it comes to synchronisation packets. I had to ensure that the first packet transmitted was all 0x00 with the sync bits 0x55 0x2d 0x16.

Next job - migrate to a Mini ITX system running CentOS5 and 1GB RAM - this could be a challenge....

Problems and Solutions

2009-02-12T10:51:00.000-08:00

Well, I decided to move the software across from CentOS 4 on an old PC to a server with 2GB of memory running CentOS 5.2.

This was 1) good because it has fixed some receive audio issues 2) bad because I've found some more problems.

Issues I have:

1) Transmit audio to a dplus reflector is fine. However, when this is onward connected to a G2 node, there is substantial break-up. My suspicion is that the G2 software can't cope with the pacing of the packets - I'm just sending them as-and-when they are available, rather than every 20ms. Dplus is fine with this - G2 obviously isn't. Solution ? Write a transmit playout buffer in the same way I had to write a receive playout buffer. More work.

2) Authentication bug. There is a state whereby the software goes into a loop, constantly sending out a re-authentication to the node.

New Feature and Change

2009-02-10T15:07:00.000-08:00

New Feature - timestamps displayed on PTT ON/OFF

Change - the "Status" enquiry has changed form " S" to " I" to maintain compatability with dplus linking on existing G2 nodes.

Future Developments

2009-02-08T09:22:00.000-08:00

1) Link to the G2 backbone. I'd like the repeater to appear in the database of other G2 nodes, and then be able to callsign route to other G2 repeaters. I'm lead to believe that this should be easy to acheive as the protocol used by dplus and G2 is virtually identical. I would need to write a listener on the gateway, and also link to the G2 database to publish my node's callsign. I would also need to update the other G2 nodes with "Last Heard" information.

2) Dual-mode working. Here's the idea. One input frequency for FM and D-Star, two output frequencies, one for FM, one for D-Star. A transcoding module (AMBE 2020 and a DAC) would transcode the audio source be-it D-Star off-air, D-Star via dplus or FM off-air to the other mode. This would enable a true, mixed-mode repeater with the minimum number of channels (I can see no way of modulating the GMSK carrier with FM baseband audio and retaining the cohesiveness of both modes.

Latest Developments

2009-02-08T08:19:00.000-08:00

Fixed - CRC is now sent correctly on all D-Plus headers.

New Feature - configuration is read from a file rather than hard-coded in the source. Not really a "feature..."

Two issues fixed, one found....

2009-02-04T06:28:00.000-08:00

Two bug fixes:

1) Loss of RX Poll was not causing node to return to "unlinked" state.

2) In the event of becoming unlinked, the node was not attempting to reconnect.

Problem found:

1) There is an error in the header CRC that is being sent to the dplus gateway. The problem is that at the moment, I am taking the CRC sent by the radio. As I am modifying the elements that make up the CRC, that invalidates the original CRC. Therefore, I need to recompute it.

New Feature - Callsign Ident

2009-02-02T02:03:00.000-08:00

Every 15 minutes, provided the node is not transmitting, an ID is sent.

If the node is on-air, then it will wait until the next break and send the ID.

The ID takes about 200ms to transmit - it is sent as a Slow Data test message, not a voice message.

New Feature - Status Message

2009-02-02T02:00:00.000-08:00

New feature:

- Put "S" in the 8th position of Your Call, and the node replies with "Not Linked," "Linked XXXXXX" and in the case of linked, with the last RX Call heard.

New Feature - GPS-A Support

2009-01-30T01:47:00.000-08:00

New Feature:

- Ability to take GPS-A mode information from IC-92 and IC-2820s and gate to APRS-IS.

The IC-91 and ID-800H only have a pass-through of NMEA GPS information - whilst is has a checksum, that is a simple XOR, and requires programming of a C1 code with it's own XOR checksum to validate the radio path.

From the IC-92, Icom introduced a feature where they format the GPS into an APRS-like string - it is much shorter than taking the entire NMEA string, and includes a proper, CCITT 16-bit CRC.

So, I now have the ability to detect the GPS-A string, check that the checksum is correct, and then gate that to APRS.

Audio Problems - Fixed

2009-01-27T13:51:00.000-08:00

Well, I've fixed the audio problem. Quality now, when connected to a repeater, is very good.

Quality when connected to a reflector is much worse, but I've some to the conclusion that that is the reflector rather than my software.

Also, the lack of a proper call-control layer on dplus has really been exercising my mind - port 20001 is used for both "call control" and the media stream, all over UDP, so if the stream goes away, trying to sense that to shut the session is difficult.

Audio problems

2009-01-27T02:46:00.000-08:00

Found a couple of bugs late last night:

1) The timing of sending the packets to the node adapter is critical. I have a routine that sends a new packet every 20ms, but there was an error as it crossed from 999ms to 1 sec. Fixed.

2) To ensure that duplicate packets and missed packets are in-filled with null voice (packets 2-20) and null sync voice (packet 1), I take the sequence number from the dplus packet, and store an entire DV frame in an array. I then read out to transmit one frame later. There is an error where the "0" ID packet in the frame is placed in the wrong part of the playout buffer. More tricky to fix, but at least I know where the problem is.

APRS-IS Working

2009-01-22T14:31:00.001-08:00

http://www.findu.com/cgi-bin/find.cgi?call=G4ULF

At last...... Wow, THAT was tough.

strktok - The WRONG thing to use

2009-01-22T13:51:00.000-08:00

Well it turns out that a) strtok is very broken and b) it isn't portable, so I've decided not to split the GPGGA and GPRMC strings that way, but to use good-old pattern matching in "while" and "for" statements. Clumsy, but portable....

strtok problems.....

2009-01-21T12:39:00.000-08:00

Now I need to split the GPGGA and GPRMC by comma delimiters and strtok is causing a segmentation fault.

Why ????? I've followed the exmples exactly - I've even tried recreating the example exactly and it doesn't work.....

Back to the drawing-board....

APRS - Questions, questions

2009-01-20T23:57:00.000-08:00

So which should I use ? GPGGA or GPRMC. They appear to be identical, except that speed and direction are included with GPRMC and Altitude in GPGGA.

The guides I've seen from people like the Cambridgeshire Repeater Group indicate that you should have both - the DPRS spec seems to indicate that GPGGA without GPRMC should be enough to send a posit.

Hmmm....

Antennae

2009-01-20T14:06:00.000-08:00

The forecast for today was very good, so I decided to take a day off and try to do something about my lack of antenna.

That and the fact that our terrestrial TV antenna has been shot-to-pieces for sometime, and just before Christmas, the satellite TV stopped as well (water in the LNB).

What a job - it should have been a 2-3 hour job; as at 10pm I have good 80 thru 6m, but no TV of any type.

And no progress on the APRS this evening :-(

APRS Interface

2009-01-20T14:00:00.000-08:00

M0VPG has loaned me his IC-E92 which has a GPS speaker/mic.

I'm experimenting with taking the GPGGA and GPRMC sentences from the DD stream and sending those to an APRS-IS server.

It turns out that the two GPS streams have a simple XOR checksum as there is no error correction on the DD stream.

Also, there is a standard for DPRS whereby you put your SSID symbol, a message and a checksum in the C1 field. This field also carries your callsign.

To send the APRS posit information, you have to be logged into the APRS-IS server - to do this, you need to use the callsign and a password generated by a hash function with a specific key.

I'm running the APRS connection as a thread so that it doesn't interfere with the running of the node/repeater, and that caused a few issues.

But, trying to get the routine to checksum the GPGGA, GPRMC and C1 messages was awful.

I'm there now, and I take the callsign from the C1 field, generate the APRS password from it and log into the APRS-IS server.

Now I need to extract the required information from GPGGA and GPRMS and create an APRS-IS posit message...

Nearly there.....

DV Header RX CallSign

2009-01-20T13:59:00.001-08:00

One slight issue - the RX Call Sign seems pretty erratic.

Sometimes you get it, sometimes you don't.

I'm wondering if it is a problem with my transmitter - I need to take a look....

Duplicate - found it....

2009-01-20T13:58:00.000-08:00

Well, I found the problem.

I was advancing whenever I received the sync frame - I wasn't checking that the sync had come from the current conversation.

Node has been working since Sunday morning on REF001C without a hitch.

Duplicate Packets

2009-01-18T03:12:00.000-08:00

Got a problem with duplicate packets.

The dplus network adds a sequence number to the packets in the stream from 0x00 to 0x14.

These then repeat, making a whole sequence 420ms.

Because I need to detect and counter out-of-order packets in the frame, I have to look for the sync sequence 0x55 0x2d 0x16, and only when I see these advance the frame buffer.

At the moment, I've got a problem where occasionally the buffer advances by several tens of frames.

I just can't find out why....

The Story So Far...

2009-01-18T03:08:00.000-08:00

I started out with:

1 x Satoshi Node Adapter board
1 x IC-91AD (yes, the US version)
1 x Ancient Dell desktop running CentOS 4
2 x Icom IC-F210S UHF FM radios
1 x FT7800R radio

and a heap of documents....

So, after a lot of work, I have turned this into a fully-working D-Star repeater and dplus node access-point.

Problems ?

Many, many, many...

About Me

2009-01-18T02:58:00.000-08:00

I've been playing with radios, computers and digital technology for years.

It started with taking two old Pye Intercoms, putting one on the back of my push-bike, connecting an antenna rather than wire, and trying to understand why I couldn't hear it.

After school, I worked for Marconi Communications Systems for two years and then went on to University. I worked as a trainee technician, fault-finding digital communications systems and HF radios.

My first foray into true digital systems was a home-built PABX designed around a Z80 CPU with a crossbar CMOS switching system. Oh, and the Morse Tutor on the ZX81....

A professional networking engineer, I have spent the last 20+ years working in the telecommunications industry since graduating with a degree in Electrical and Electronics Engineering from City University, London.

I am a Chartered Engineer, hold a CCIE in Routing and Switching, can program in C++ (very badly, but it works). My day job is as a senior consultant in Unified Communications technologies, mostly around interoperability.

I've held an Amateur Radio licence since 1984 - originally G6TBA, then took the dreaded CW and became G4ULF. This enabled me to run my school's Amateur Radio station on HF and put a vertical in the back-garden of my parent's house, knocking out most TVs within a 2 mile radius....

Married with a 7-year old son, I am Chairman of the Ashdown Forest Repeater Group and a member of the local club, Crawley Amateur Radio Club.

I enjoy working HF mobile (not much chance at home), and on IRLP and EchoLink.

73s de G4ULF

Welcome to my Blog....

2009-01-18T02:53:00.000-08:00

Well, in my opinion, D-Star is the most exciting thing to happen to Amateur Radio in decades.

But, I hate having to pay anyone for anything - especially if I don't get a choice of suppliers. So, whilst I am a fan of digital communications, (I do it for a living !), I am NOT a fan of any protocol that is not truly open.

So, I decided to see what I could do with a supposed "closed" protocol, D-Star.

What I discovered pretty early on is that although there is only one current manufacturer, Icom, only the voice Codec is closed; AMBE-2020 from DVSI.

So, provided you don't touch the audio in the payload, game on....

This Blog is going to be my running log of development of a D-Star repeater that links to the worldwide dplus network running on homebrew components and standard UHF FM rigs.

Welcome on-board !

David G4ULF

Linux DStar Development

New Project Name

NI-Star v2 has arrived

Announcing the Imminent Arrival of NI-Star

Why so quiet ?

Dayton 2010

Release Plans

v6 Test System on-line

Logging changes - future support

G2_poll bug

RadCom Article on GB7MH

Latest Changes

2010 - Happy New Year

Bug/Feature and v6 update

v6 Beta

Bugs....

Progress with the v2.06 board !

S/N Board Progress

To COS or not to COS - that is the question...

Update - 31/10/09

And the next bug is...

G2 Update/Routing Issues Fixed

GB7MH D-Star Improvements

GB7BM QRV

Bugs, bugs, bugs

GB7MH QRV

August - slow month

Fully Operational

Never use Global Variables.....

Grrr - Postgres

SQLite is no more

Live QSO - G4ULF to G3SMU

Linux G2 Announcement

G2 Station Registration

Tait T800 and GMSK

G2 Tests Update

Update - end of June 2009

D-Star Monitor

How cool is this ???

D-Star Monitor

Update - June 2009

G2 Integration Testing

G2 Migrated to SQLite

Progress Report

4.17 At Last; and a few other code changes

Satoshi 4.xx Code

"Send Messge" bugs

RX Audio Improvements

Satoshi 4.06 Code

More RX Audio Problems

Audio Problem Investigations...

Audio Problems - again

Progress on G2

Bug Fixes

QSL VA3UV on XRF005 B

Bug Fix and testing progress

Bug Fix/Major New Feature

Bug Fixes/New Features

Testing Outcome

Transmit re-work

Problems and Solutions

New Feature and Change

Future Developments

Latest Developments

Two issues fixed, one found....

New Feature - Callsign Ident

New Feature - Status Message

New Feature - GPS-A Support

More problems fixed

Audio Problems - Fixed

Audio problems

APRS-IS Working

strktok - The WRONG thing to use

strtok problems.....

APRS - Questions, questions

Antennae

APRS Interface

DV Header RX CallSign

Duplicate - found it....

Duplicate Packets