Jump to content


Photo

Storage / workflow


  • Please log in to reply
8 replies to this topic

#1 M Rose

M Rose

    Advanced Member

  • Members
  • PipPipPip
  • 1,918 posts
  • LocationMelbourne, Australia

Posted 11 August 2019 - 02:51 AM

I have an external hard drive that just died and had a lot of old course files on it. While waiting to see if I can recover it, I'm backing up all my existing stuff to cloud storage and it is taking FOREVER, like 2 hours for ONE scene folder.

 

I've used up 90 percent of my desktop storage space and obviously I need to clear stuff out, but I'm also finding myself wondering how much of it is wasted by having a copy of the same files in every single folder. All the PP files, all the PP library files, all the same Assets, pretty much all the same trees, textures etc..

 

Is anybody working with one default Unity Project and able to re-use it over and over? I assume switching out scenes and terrains is workable.

 

Pretty much every course project folder is pushing 50-60 GB and it's really becoming a major PITA to move them around to other drives and directories.

 

It also occurred to me - if I just move the scene, library, and terrain files over to a separate folder, do I need to copy anything else? Obviously it would save tons of time and space if that's all I needed to keep. Importing those three things into a separate project should theoretically keep things intact, right?


VG2 • PG • 2k21 • 2019 • Links • GBC • JN5


#2 Tresclub

Tresclub

    Advanced Member

  • Members
  • PipPipPip
  • 636 posts
  • LocationLa Mesa, CA USA

Posted 11 August 2019 - 06:20 AM

Matt,  Have you thought about investing in a 500GB SSD?

 

https://www.amazon.c...ics,212&sr=1-1]

Samsung SSD


PerfectGolfMember2.jpg 

“You only live once, do it right, and once is enough."

 

Windows 10 Home 64-bit |Intel® Core™ i7-7700K CPU @ 4.20GHz (8 CPUs), ~4.2GHz | nVidea GeForce GTX 1070 FTW
16GB Corsair Vengeance LPX DDR4 | 2/Samsung EVO 850 500gb SSD | Corsair 600cx PwrSup
BenQ ZOWIE 27" 1ms LCD HD | DirectX 12 | Realtek High Definition Audio | HD Pro Webcam C920


#3 jspirate

jspirate

    Advanced Member

  • Members
  • PipPipPip
  • 487 posts

Posted 11 August 2019 - 02:45 PM

These go onsale from time-to-time (techbargains) and if you have USB 3.1 they are lighting quick.  I got the 500GB version and have nothing but my courses on it.  For my current WIP I alternate saving between my non-system internal drive then the SanDisk (back and forth).  That way I always have some version of it on two different drives.

 

Its amazing how fast the drive is when using USB 3.1.  It does come with and adapter cable if you don't have 3.1.

 

https://www.amazon.c...le?ie=UTF8&th=1

 

 

I copy the whole folder... Its big, but it seems to work well when you need to recover.


i7-8700K
GIGABYTE Z390 AORUS Pro
64 GB DDR4 3200
MSI GeForce GTX 1080 Ti Gaming X
LG 32GK850G
Win 10 Home Premium on 1TB SSD

 

Courses:

Bradford Links - released

Ironwood - released

Cypress Landing - released (also available in a 3DG version)

Crow's Nest - released


#4 Stingreye

Stingreye

    Advanced Member

  • Members
  • PipPipPip
  • 341 posts
  • LocationSouthern California

Posted 11 August 2019 - 04:03 PM

I have an external hard drive that just died and had a lot of old course files on it. While waiting to see if I can recover it, I'm backing up all my existing stuff to cloud storage and it is taking FOREVER, like 2 hours for ONE scene folder.

I've used up 90 percent of my desktop storage space and obviously I need to clear stuff out, but I'm also finding myself wondering how much of it is wasted by having a copy of the same files in every single folder. All the PP files, all the PP library files, all the same Assets, pretty much all the same trees, textures etc..

Is anybody working with one default Unity Project and able to re-use it over and over? I assume switching out scenes and terrains is workable.

Pretty much every course project folder is pushing 50-60 GB and it's really becoming a major PITA to move them around to other drives and directories.

It also occurred to me - if I just move the scene, library, and terrain files over to a separate folder, do I need to copy anything else? Obviously it would save tons of time and space if that's all I needed to keep. Importing those three things into a separate project should theoretically keep things intact, right?

Probably internet related, I have all my files backed up to both one drive’s cloud and acronis true image cloud simultaneously and it works seamlessly for me. My connection is a 150/150 frontier FIOS line. If your internet line is fast, you very well might be getting throttled to certain cloud storage providers. I used to use carbonite and they started throttling things so initial backups seemed to take forever but new stuff wasn’t too bad.

#5 M Rose

M Rose

    Advanced Member

  • Members
  • PipPipPip
  • 1,918 posts
  • LocationMelbourne, Australia

Posted 11 August 2019 - 06:36 PM

I'm not really looking for hard drives; I'm more interested in making my course files and scenes more economical.

 

My average course folder is about 10-15 GB and has roughly 50,000 files in it, and 80 percent of those files I probably have copies of in every single course folder (trees, libraries, standard assets, etc).

 

Is anyone able to just design in one master Unity project without needing to copy 10 GB worth of crap everytime they start a new one? I guess I feel like I'm wasting a lot of time and space, as said before.

 

Also when it comes to backing up, it is the same issue - I realized I'm backing up the same stuff over and over again with the only difference being a few things like the terrain, the scene, and the flag, splash, etc..

 

 


VG2 • PG • 2k21 • 2019 • Links • GBC • JN5


#6 Joe Habiger

Joe Habiger

    Advanced Member

  • Members
  • PipPipPip
  • 1,037 posts
  • LocationCoon Rapids, MN

Posted 11 August 2019 - 10:48 PM

Matt I have never tried this but yeah that almost seems like it would work because if you keep all the same objects and such in a folder all you would need is to copy over the stuff you talked about and then just open the scene file and continue working on that one. I would imagine you could have all the .scene files, flags and splash screen in one folder if they are all named differently and just move each Terrain.asset file because that is the only thing you can't rename but you could I believe back it up with a different name and when opening the scene it will probably go to that terrain file and load that as well?

 

I have not tried this stuff but wow would that save a ton of room.


  • AMD Ryzen 7 5800X
  • Asus ROG Strix B550-F Gaming
  • Sapphire Nitro+ SE AMD Radeon 6800XT
  • G.Skill Trident Z Royal RGB 16GB DDR4-3600
  • Sabrent 2TB Rocket NVMe 4.0 Gen 4 PCIe M.2
  • Samsung 970 EVO Plus 1TB SSD M.2 NVMe
  • Samsung 850 EVO 512GB SSD
  • Sound Blaster ZX Soundcard
  • EVGA 750 Gold Power Supply
  • Fractal Design Meshify S2 Case
  • Cooler Master MasterLiquid ML360R
  • "43 inch Vizio 4k Monitor

#7 clubcaptain

clubcaptain

    Advanced Member

  • Members
  • PipPipPip
  • 1,310 posts

Posted 12 August 2019 - 12:00 AM

Matt I have never tried this but yeah that almost seems like it would work because if you keep all the same objects and such in a folder all you would need is to copy over the stuff you talked about and then just open the scene file and continue working on that one. I would imagine you could have all the .scene files, flags and splash screen in one folder if they are all named differently and just move each Terrain.asset file because that is the only thing you can't rename but you could I believe back it up with a different name and when opening the scene it will probably go to that terrain file and load that as well?

 

I have not tried this stuff but wow would that save a ton of room.

Not sure how a ton of room would be saved. All the assets would still be there -just brought from a single source instead of many. There again, not all assets are used on every course so someone could end up with many more files than were necessary even if they were quick to install. For example, I choose specific Speedtree trees for each project. I don't want them all in every project. Same with grasses and bushes and buildings and everything else. If doing a links course no trees may be needed.


PC specs...

Intel core I7 9700k 3.6 GHZ-Turbo 4.9 GHZ
64 GB Corsair vengeance  LPX DDR4 2400 MHZ
Asus PRIME Z390-P 
Nvidea GeForce RTX 2060 6GB
X box 360 wired controller
Windows 10 PRO NA 64 bit


#8 M Rose

M Rose

    Advanced Member

  • Members
  • PipPipPip
  • 1,918 posts
  • LocationMelbourne, Australia

Posted 12 August 2019 - 12:37 AM

I tend to re-use most of the same trees and bushes so that wouldn't be a huge issue for me.

 

I actually tried merging all the desert courses into one scene set and it seems to have worked; maybe instead of one master, I could try a small group of the tied to different styles.


VG2 • PG • 2k21 • 2019 • Links • GBC • JN5


#9 Joe Habiger

Joe Habiger

    Advanced Member

  • Members
  • PipPipPip
  • 1,037 posts
  • LocationCoon Rapids, MN

Posted 12 August 2019 - 01:35 AM

Not sure how a ton of room would be saved. All the assets would still be there -just brought from a single source instead of many. There again, not all assets are used on every course so someone could end up with many more files than were necessary even if they were quick to install. For example, I choose specific Speedtree trees for each project. I don't want them all in every project. Same with grasses and bushes and buildings and everything else. If doing a links course no trees may be needed.

 

Correct with all the objects issue but you don't have to use them. Again I have not tried this so I have no idea if it works but instead of the 50 GB's I have with 4-5 courses in my folder I keep all my unity designs I can have one 10 GB folder with everything even if I don't use this certain tree or not. With all the crap I accumulate sometimes even with (2 SSD drives and 1 external data drive) it could be worth it especially when I keep adding to my JNPG collection which is now at 45 GB and I still am missing lots of courses I have not downloaded yet..lol

 

40 GB saved right there....


  • AMD Ryzen 7 5800X
  • Asus ROG Strix B550-F Gaming
  • Sapphire Nitro+ SE AMD Radeon 6800XT
  • G.Skill Trident Z Royal RGB 16GB DDR4-3600
  • Sabrent 2TB Rocket NVMe 4.0 Gen 4 PCIe M.2
  • Samsung 970 EVO Plus 1TB SSD M.2 NVMe
  • Samsung 850 EVO 512GB SSD
  • Sound Blaster ZX Soundcard
  • EVGA 750 Gold Power Supply
  • Fractal Design Meshify S2 Case
  • Cooler Master MasterLiquid ML360R
  • "43 inch Vizio 4k Monitor




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users