Category PS4 CFW and Hacks       Thread starter PSXHAX       Start date Jul 17, 2018 at 1:23 AM       119,840       396            
Earlier this month we saw a PS4Updates Python script, and recently PlayStation 4 developer GarnetSunset (Twitter) made available a PlayStation Store DLC Index script tool called psDLCIndex which can be used to scrape / download PS4 PKG downloadable content. :D

Download: psDLCIndex-master.zip / GIT / How to Change CUSA of DLC to Match CUSA of Game Guide / 0-byte PSN DLC Downloader by Etheldreda-MU / psDLC.zip / psDLC (c# version) by stooged / fake_dlc_pkg.rar (0.3 MB) / Playstation Store DLC Indexer.rar (3.69 MB) by pearlxcore / cindexTutorial.rar (1.23 MB) by ohcHIT

:alert: From Al Azif, to quote: "THESE TOOLS WILL NOT WORK WITH MOST DLC. The DLC must be an unlockable type with no extra data required and may require a specific game update version. There is no way to know 100% if it'll work until you try it."

:arrow: For those having trouble, see this Workaround by Baku86 to fix the issue. :thumbup:

This comes following the tutorial for Dumping PS4 5.05 DLC, Games, Updates & Themes with details below from the README.md to quote: psDLCIndex

PlayStation Store DLC Indexer, and list generator.

What is this?

This is a script that will go through a PlayStation store entry for a certain title, which is given at runtime, and get the titleIDS of all DLC associated with them. This can then be funnelled into "mysteriouslink.py" which can parse it out and, using some outside programs, generate some cool pkgs.

How to use?


Make sure you run "requirements.bat" if you're on windows, but then, just run the "dlcIndex.py" script and input the full title ID of the app you wish to crawl. This can take a while so be patient.

When done, make sure to run mysteriouslink, with all the files required, which I can't link, in this directory. You should have a bunch of PKGs.

HOW TO GET PS4 DLC GAME (COMPATIBLE WITH FAKE PKG GAMES) via kepz90

HOW TO PATCH ORIGINAL PS4 DLC THEME (SEMI PERMANENT) via kepz90
:arrow: Update: In related news, windsurfer1122 made available PSN_get_pkg_info which is a Python script to extract package information from header and PARAM.SFO of PS3 / PSX / PSP / PSV / PSM and PS4 packages.

Download: PSN_get_pkg_info-master.zip / GIT

And from the README.md, to quote: PSN_get_pkg_info.py (c) 2018 by "windsurfer1122"

Extract package information from header and PARAM.SFO of PS3/PSX/PSP/PSV/PSM and PS4 packages.

Goals:
  • One-for-all solution to retrieve all header data and PARAM.SFO data from PSN packages
  • Decryption of PS3 encrypted data to get all data
  • Support of all known package types: PS3/PSX/PSP, PSV/PSM, PS4
  • Easy enhancement of interpreting data (=done at the very end with all data at hand)
  • Support multiple output formats
  • Support multiple debug verbosity levels
  • Easy to maintain and no compiler necessary (=interpreter language)
  • Cross platform support
    • Decision: Python 3
      • Compatible with Python 2 (target version 2.7)
        • Identical output
        • Forward-compatible solutions preferred
  • Modular and flexible code for easy enhancement and/or extensions (of course there's always something hard-coded left)
For options execute: PSN_get_pkg_info -h
Use at your own risk! If you state URLs then only the necessary bytes are downloaded into memory.

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see https://www.gnu.org/licenses/.

git master repository at https://github.com/windsurfer1122

:arrow: Update #2: Developer lucasepe made available another scraping utility called Go PS4 which allows you to search your favorite PS4 games from PlayStation Store using the Command Line, which details below from the README.md to quote:

Download: go-ps4-master.zip / GIT

This tool use Colly the Lightning Fast and Elegant Scraping Framework for Gophers

How to build

Windows

Code:
go build -ldflags "-X main.version=0.0.1 -X main.date=%date:~-4%-%date:~3,2%-%date:~6,2%T%time:~0,2%:%time:~3,2%:%time:~6,2%"
Linux
Code:
go build -ldflags "-X main.version=0.0.1 -X main.date=%date:~10,4%-%date:~4,2%-%date:~7,2%T%time:~0,2%:%time:~3,2%:%time:~6,2%"
How to use

List available options

Code:
go-ps4 -help
Usage of go-ps4:
  -addons
       show also extra contents
  -free
       show only free titles
  -lang string
       language (it, en) (default "it")
  -search string
       search for specified title
  -version
       show app version
  -weekly-deals
       show only weekly deals titles
Search only for free games
Code:
go-ps4 -free
Search for a specific game
Code:
go-ps4 -search "strange brigade"
Show only weekly deals
Code:
go-ps4 -weekly-deals
PSDLCIndex PlayStation Store PS4 DLC Indexer by GarnetSunset.jpg
 

Comments

demonfish

Senior Member
Contributor
Verified
Forget about overrated and what not... I need some kind soul with knowledge to just answer my simple questions please. ;) I mean ...how hard can that be ? ... :D

I am a gutted for trying things I know little about... This is no exception. :p
 

biibop

Senior Member
Contributor
WIP by Etheldreda-MU:


he's got his own version of the script on github: main.py
Code:
#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import argparse
import datetime
import json
import os
import urllib3
import re
import shutil
import subprocess
import sys
import time

import certifi

CWD = os.path.dirname(os.path.realpath(__file__))
TEMP = os.path.join(CWD, 'tmp')
OUTPUT = os.path.join(CWD, 'output')
PKGTOOLS = os.path.join(CWD, 'tools')


def safe_delete(path):
    if os.path.isfile(path):
        os.remove(path)


def bootstrapped():
    try:
        fail = False
        if not os.path.exists(TEMP):
            os.mkdir(TEMP)
        else:
            for root, dirs, files in os.walk(TEMP):
                for f in files:
                    os.unlink(os.path.join(root, f))
                for d in dirs:
                    shutil.rmtree(os.path.join(root, d))
        if not os.path.exists(OUTPUT):
            os.mkdir(OUTPUT)
        if not os.path.exists(PKGTOOLS):
            os.mkdir(PKGTOOLS)
        if not os.path.exists(os.path.join(PKGTOOLS, 'ext')):
            os.mkdir(os.path.join(PKGTOOLS, 'ext'))
    except (OSError, PermissionError):
        print('Could not create necessary directories')
        fail = True

    if not os.path.isfile(os.path.join(PKGTOOLS, 'orbis-pub-cmd.exe')):
        print('Missing "orbis-pub-cmd.exe" from tools directory!')
        fail = True

    if not os.path.isfile(os.path.join(PKGTOOLS, 'ext', 'sc.exe')):
        print('Missing "ext\\sc.exe" from tools directory!')
        fail = True

    if not os.path.isfile(os.path.join(PKGTOOLS, 'ext', 'di.exe')):
        print('Missing "ext\\di.exe" from tools directory!')
        fail = True

    if fail:
        return False

    return True


def build_header(lang, region):
    return {
        'Host': 'store.playstation.com',
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0',
        'Accept': '*/*',
        'Accept-Language': '{}-{}'.format(lang, region),
        'Accept-Encodings': 'gzip, deflate, br',
        'Connection': 'keep-alive',
        'Pragma': 'no-cache',
        'Cache-Control': 'no-cache'
    }


def get_json(http, url, header):
    response = http.request('GET', url, headers=header)

    return json.loads(response.data.decode('utf-8'))


def validate_cid(cid):
    cid = cid.upper()

    if len(cid) == 9:
        cid = '{}_00'.format(cid)
 
    if len(cid) > 36:
        match = re.search(r'([A-Z]{2}[\d]{4}\-[A-Z]{4}[\d]{5}\_00\-[A-Z\d]{16})', cid)
        if match:
            return match.group(1)

    if re.match(r'^[A-Z]{2}[\d]{4}\-[A-Z]{4}[\d]{5}\_00\-[A-Z\d]{16}$', cid) or re.match(r'^[A-Z]{4}[\d]{5}\_00$', cid):
        return cid
 
    return ''


def build_SFX(sfxFile, title, cid):
    pattern = re.compile(r'[\W_]+', re.UNICODE)
    safeTitle = pattern.sub('', title)
    sfx = '''<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<paramsfo>
  <param key="ATTRIBUTE">0</param>
  <param key="CATEGORY">ac</param>
  <param key="CONTENT_ID">{}</param>
  <param key="FORMAT">obs</param>
  <param key="TITLE">{}</param>
  <param key="TITLE_ID">{}</param>
  <param key="VERSION">01.00</param>
</paramsfo>'''.format(cid, safeTitle, cid[7:16])

    with open(sfxFile, 'wb+') as buf:
        buf.write(bytes(sfx, 'utf-8'))


def build_GP4(gp4File, cid, sfoFile):
    generationTime = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
    gp4 = '''<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<psproject fmt="gp4" version="1000">
  <volume>
    <volume_type>pkg_ps4_ac_nodata</volume_type>
    <volume_id>PS4VOLUME</volume_id>
    <volume_ts>{}</volume_ts>
    <package content_id="{}" passcode="00000000000000000000000000000000"/>
  </volume>
  <files img_no="0">
    <file targ_path="sce_sys/param.sfo" orig_path="{}"/>
  </files>
  <rootdir>
    <dir targ_name="sce_sys"/>
  </rootdir>
</psproject>'''.format(generationTime, cid, sfoFile)

    with open(gp4File, 'w+') as buf:
        buf.write(gp4)


def main():
    if bootstrapped():
        parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.ArgumentDefaultsHelpFormatter)
        parser.add_argument('contentIDs', nargs='+', help='Content ID/Store URL to create PKGs for')
        parser.add_argument('-r', '--region', type=str, required=False, default='en/US', help='Region to search, will not effect full URLs')
        args = parser.parse_args()

        region = args.region
        if not region or not re.match(r'[a-zA-Z]{2}\/[a-zA-Z]{2}', region):
            print('Invalid Region!')
            return

        http = urllib3.PoolManager(cert_reqs='CERT_REQUIRED', ca_certs=certifi.where())
        header = build_header(region.split('/')[0].lower(), region.split('/')[1].upper())

        for cid in args.contentIDs:
            if len(cid) > 36:
                match = re.search(r'^http[s]{0,1}\:\/\/store\.playstation\.com\/([a-zA-Z]{2}\-[a-zA-Z]{2})\/', cid)
                if match:
                    region = match.group(1).replace('-', '/')
            cid = validate_cid(cid)

            if not cid:
                print('Invalid Content ID!')
                continue

            url = 'https://store.playstation.com/valkyrie-api/{}/999/resolve/{}'.format(region, cid)
            try:
                data = get_json(http, url, header)
            except json.decoder.JSONDecodeError:
                print('Error Decoding JSON!')
                continue
            try:
                actualCid = data['included'][0]['id']
                actualTid = actualCid[7:19]
            except KeyError:
                print('Error checking for redirect')
                continue

            if cid != actualCid and cid != actualTid:
                if len(cid) == 12:
                    actual = actualTid
                else:
                    actual = actualCid
                print('Expected {}, but found {}, skipping PKG creation'.format(cid, actual))
                continue

            for entry in data['included']:
                try:
                    if entry['attributes']['kamaji-relationship'] == 'add-ons' and (not entry['attributes']['file-size']['value'] or (entry['attributes']['file-size']['unit'] == 'KB' and entry['attributes']['file-size']['value'] < 1240)):
                        sfxFile = os.path.join(TEMP, entry['id'] + '.sfx')
                        sfoFile = os.path.join(TEMP, entry['id'] + '.sfo')
                        gp4File = os.path.join(TEMP, entry['id'] + '.gp4')
                        pkgFile = os.path.join(OUTPUT, entry['id'] + '-A0000-V0100.pkg')

                        try:
                            build_SFX(sfxFile, entry['attributes']['name'], entry['id'])
                            build_GP4(gp4File, entry['id'], sfoFile)

                            orbisPub = os.path.join(PKGTOOLS, 'orbis-pub-cmd.exe')

                            try:
                                with open(os.devnull, 'w') as fnull:
                                    print('Creating {}...'.format(sfoFile.replace(TEMP + '\\', '')))
                                    subprocess.check_call([orbisPub, 'sfo_create', sfxFile, sfoFile], stdout=fnull)
                                    print('Creating {}...'.format(pkgFile.replace(OUTPUT + '\\', '')))
                                    subprocess.check_call([orbisPub, 'img_create', gp4File, pkgFile], stdout=fnull)
                            except subprocess.CalledProcessError:
                                print('Unable to create SFO/PKG!')

                            safe_delete(sfxFile)
                            safe_delete(sfoFile)
                            safe_delete(gp4File)
                        except KeyboardInterrupt:
                            safe_delete(sfxFile)
                            safe_delete(sfoFile)
                            safe_delete(gp4File)
                            safe_delete(pkgFile)
                            print('Canceling!')
                            return 0
                except KeyError:
                    pass

        print('Done!')
        return 0

    return 1


if __name__ == '__main__':
    main()
0-byte PSN DLC Downloader

Requirements
  • Windows
  • Python3 with urllib3 and certifi
  • Fake PKG Tools (tools/orbis-pub-cmd.exe, tools/ext/sc.exe, tools/ext/di.exe)
How to use
  • Download main.py
  • Create a tools directory in the same place as main.py and place the Fake PKG tools in it
  • Run in the command line with python main.py X
    • X can be the Title ID (CUSA05258), Content ID (UP0700-CUSA05258_00-PS4TOBERSERIA001), or the store URL (https://store.playstation.com/en-us/product/UP0700-CUSA05258_00-PS4TOBERSERIA001)
    • You may specify a region with -r ja/JP the region is only used with the Title/Content ID. The region is taken from the URL when using a URL input.
      • Default if not specified is en/US
    • You may specify more than one input python main.py CUSA05258 CUSA2510, however you may only specify one region at a time (Unless using URL input as regions are pulled from the URL)
  • PKG files will be created in output
Credits
  • TheRadziu - Original Concept
  • Al-Azif - API Info and Python Help
 

barelynotlegal

Senior Member
Contributor
Verified
I can confirm it works. WOFF working, added my collesium tickets. Did the last of us and under notifications everything installed but my game is base so no way of knowing. Doing a few other games and will see if it works still.

All I do is use pkgviewer then sfo editor to get the correct tittle is. Then I use the dlc.py then I drag and drop the txt file into mysterious and it works. (I copied and pasted the integrity of my fpkg generator)
 

demonfish

Senior Member
Contributor
Verified
@barelynotlegal
How do you input the Title ID into the python script mate? If you could paste the cmd TEXT format you use.? I am sure this is where I am going wrong. I am sure I must be putting the title ID in the wrong way ? eg: does it include \ or other unique keystrokes etc.. ?

No wonder I can not get this to work correct ....I justed watched that video clip and when I follow this method ...the script insta-closes unlike the one in the youtube clip. WTF
What can be the problem? A/V blocking ? maybe.

Thanks for any help or advice.
Best Regards,
 
Recent Articles
GOG Galaxy 2.0 AIO Video Game Launcher Open Beta Arrives
Along the lines of popular emulator frontends such as LaunchBox, recently the GOG Galaxy 2.0 (aka Good Old Games) all-in-one video game launcher Open Beta went live on GOG.com bringing features...
Sony PlayStation 4 & PS VR Last-Minute Holiday Deals at PSDirect.com
Following the launch of their online store, today Sony shared some last-minute holiday deals for PlayStation fans featuring savings on PS4, PS VR, select titles and more at their PSDirect.com...
Microsoft Xbox Series X Coming Holiday 2020, Power Your Dreams!
Formerly known under the codename Project Scarlett, today Microsoft officially announced their Xbox Series X next-generation console at The Game Awards scheduled to launch alongside Sony's...
Godfall PS5 Reveal Trailer, Bring on the Next-Gen PlayStation 5 Scene!
Like many when first seeing this, I did a double take to make sure my eyes were really seeing a PS5 game trailer... lo and behold it actually is a PlayStation 5 game trailer for the upcoming...
Top