Help with network performance

From: Nick Smith 
------------------------------------------------------
Im trying to figure out some networking issues on my home network and
id like to see if any of the network guru's on the list can give me
any advice.

Im using FreeNAS as my storage solution using ISCSI to map a drive to
my media center PC (windows 7)

I have dedicated gigE network cards and am using a crossover cable to
connect the two machines. (taking my gig switch out of the loop to see
if i get better speeds)

Have static IPs setup on each nic. 192.168.1.2 and 192.168.1.3

I did some tests coping a 6.5gig file to and from the boxen

Writing From the Win7 box to FreeNAS coping 6.5 gigs of data it starts
out around 130MB/sec and finally settles around 86MB/sec by the time
its finished
(just going by what windows its telling me)
Ive been playing with jumbo frames to try and get faster transfer rates.

Time are below:

W/O jumbo frames
1:53.9

W jumbo frames (7K on both)
1:55.5

W jumbo frames (freenas with 9K, win with 7K, nic on win box doesnt
support higher than 7K)
1:37.5

Writing From the FreeNAS to the Win7 box same file transfer
Speeds start our around 12MB/sec and work their way up to around
17MB/sec with jumbo frames on.

W/O jumbo frames
Speeds start out around 20MB/sec and work their way up to 31MB/sec
4:23.4

W jumbo frames (7K)
6:36.5

W jumbo frames (nas with 9K, win with 7K)
6:13.2

Im really not sure what speeds im suppose to be getting.

I use FreeNAS for dedicated media storage and stream movies/music off
the nas using media center/media browser on Win7.

As you can see the speeds are alot better writing to FreeNAS than
reading from it.
And as such my movies skip and stutter as they play, which is what im
tring to resolve.

I always thought reading from a raid was alot faster than writing to it.
(6 x 2TB disks on an IBM Serve raid in this case)
Thats why i think its something with my network.

Jumbo frames give me better write to FreeNAS speeds, but it ends up
hurting me reading from it for some reason. (even with both set to the
same size 7K)

I have a dlink green gigE switch that everything else on my network is
connected to, and thought i would get better performance if i took
that out of the loop and went with a straight crossover cable.
I wanted to eliminate other network traffic, wireless etc from the
mix, and from what i read you can use jumbo frames without a switch.

I really want to increase my network read speeds from FreeNAS to try
and fix the jitter/stutter so i can actually stream a movie off the
thing.

Any ideas on how to tune this setup to give me more speed reading from FreeNAS?

My ifconfig from FreeNAS:

bce1: flags=8843 metric 0 mtu 7000
        options=c01bb

=============================================================== From: John Aldrich ------------------------------------------------------

=============================================================== From: Phil Sieg ------------------------------------------------------ Nick, Don't know how much this will help... I have 6TB raid 5 enclosure (4 2tb disks, software raid) that is attached to my home server running ubuntu 10.04 by USB 2.0. I can stream HD content (720p, not sure if I have any 1080...) to any of my 5+ computers or devices (wired, or wireless) with no hiccup or stutter. Unfortunately I have no Idea of what my read/write speeds are. I am using the 6TB disk as a network drive and the RAID function is built in to the enclosure. Phil Sieg President SeniorTech LLC / Snapfon www.snapfon.com B: 423.535.9968 F: 423.265.9820 M: 423.331.0725 phil.sieg@seniortechllc.com

=============================================================== From: "Alex Smith (K4RNT)" ------------------------------------------------------ Can't say, iSCSI from what I understand requires a lot of CPU overhead, you may want to use SMB or NFS instead, will probably give you better transfer rates. 000

=============================================================== From: "Alex Smith (K4RNT)" ------------------------------------------------------ John, you have your RAID 1 and RAID 0 backwards, RAID 0 is striping, increased throughput, no redundancy; RAID 1 is mirroring, redundancy, no change in performance, in some circumstances... . .

=============================================================== From: John Aldrich ------------------------------------------------------ Quoting "Alex Smith (K4RNT)" : I sit corrected. Thanks. At least I didn't pretend I knew what

=============================================================== From: John Aldrich ------------------------------------------------------ Quoting "Alex Smith (K4RNT)" : hehe... At least I didn't pretend I knew what I was talking about. :D Thanks for the correction

=============================================================== From: Billy ------------------------------------------------------ As others pointed out, you maybe CPU bound. Also, I know that some cards don't always negotiate the best possible link when they are x-over. Some cards are kind of dumb and rely on the buffer and smarts of the switch they are connected to. Also, your windows read speed might be due to some weird windows "sync write when reading from network" or some other type of foolishness that M$ felt they should "fix" for you. Can you write a python script that just reads and throws away the file buffer.. Rather than actually write it to disk? Sent from my iPhone

=============================================================== From: "Alex Smith (K4RNT)" ------------------------------------------------------ I would suggest you reconnect the switch. You may want to look at the specifications of the switch, and verify that both the switch, and all the NICs involved (sender and recipient) support jumbo frames. This can sometimes cause some bottlenecks and slow down the transfer. Also, encryption causes a CPU bottleneck, requiring to encrypt the data and then send it, can slow down the transfer, sometimes up to 10x! Wish you luck on getting this working. Let us know what your ultimate solution is! k $ . .

=============================================================== From: James Nylen ------------------------------------------------------ Another way to test the network hardware only is iperf (sets up a client/server connection and just shoves a bunch of data through it without saving it or anything). Pretty sure it's Linux only though.

=============================================================== From: Dave Brockman ------------------------------------------------------ -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I can only speak for myself, but I must admit to not noticing the default reply-to behavior of the list recently changed. Others may need or want to take notice of this as well.... This mimics my personal benchmarking.... FreeNAS (7 and 8) iSCSI was pretty abysmal. Paraphrasing for list brevity... Well, part of your problem is your GigE card in a PCI bus. I understand this may be a limitation of your hardware, but you're never going to get real Gig speeds out of a shared PCI bus. I also pretty much avoid RTL as a general rule of thumb, but I'm pretty sure that's also the cheapest GigE card recognized by VMWare, for times you're in a pinch for a demo or rescue system and just... need... esx... to... find... a... damn... hdc... for... storage... and... a... gig... nic... I suspect the hardware here is not the problem. If the NICs are on a PCI bus, they should be on a dedicated bus in that hardware. Try different cables. If/When you bring switch back into the equation, the ports that talk iSCSI should be in a different VLAN from your other traffic. Just putting them on a different IP subnet does not dedicate switch fabric as storage fabric. Reference above regarding PCI busses, but if that's all you have to work with, an Intel Pro should be a safe bet. I'm more of an ebay hardware type of guy, but you know :) Controller hardware is probably not your issue. Drive choice isn't helping you any, but probably not the main issue. They aren't dropping out going to sleep while you're actively copying data to them.... No, just putting them on a different IP subnet without isolating them via VLAN still shares the same layer 2 broadcast domain with the rest of your network, which if it includes wireless is quite chatty. HDD's are cheap... :) Regards, dtb - -- "Some things in life can never be fully appreciated nor understood unless experienced firsthand. Some things in networking can never be fully understood by someone who neither builds commercial networking equipment nor runs an operational network." RFC 1925 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAlA8NwIACgkQABP1RO+tr2QLyQCfZzgt3oONcUBxDd6VPeMhCt2S a/MAoLbf9ChXg328NIfEMi9iRm/S4mvZ =vtbg -----END PGP SIGNATURE-----